More documentation. LICENSE and CCL.

[egate.git] / doc / egate.sgml

<!doctype linuxdoc system>

<!--
  $Id: egate.sgml,v 1.3 1995/07/05 10:06:42 adam Exp $
-->

<article>
<title>Email/Z39.50 gateway guide
<author>Europagate, 1995
<date>$Revision: 1.3 $
<abstract>
This document describes a Email server that provides access to the
Z39.50 protocol.
</abstract>

<toc>

<sect>Introduction

<p>
This document describes how to compile, install and setup the
Email server (ES) software. It does not address the internal design of
the software.

<sect>Before you begin

<p>
An ANSI C compiler is required in order to compile the ES software.

The ES can use either CNIDR's zdist package or the YAZ package from
Index Data to interface the Z39.50 protocol. So you need to obtain
either of these first.

The Zdist package can be found in:

<url url="ftp://ftp.cnidr.org/pub/NIDR.tools/zdist/zdist102b1-1.tar.Z" >

The zdist doesn't support result-set references. Also, it has a few
bugs &mdash; see the file zdist.es.patch for details.

YAZ can be found in:

<url url="ftp://ftp.algonet.se/pub/index/yaz/">.

The ES also use GNU's regex package to parse regular expressions. 
The ES has been tested with regex-0.12. Some systems, such as Linux,
comes with the regex package preinstalled.

<sect>Compilation

<p>
Unpack <tt>egate.tar.gz</tt> and edit the top level <tt/Makefile/. Specify
where the GNU regex package is located and specify whether you use
YAZ or zdist. One some systems, you may have to set the <tt/NETLIB/ as
well.

You may wish to set <tt/CC/ and <tt/CFLAGS/ in your shell, since these
will affect the compilation &mdash; these are not set in the <tt/Makefile/.

Now, type <tt/make/. 

<sect>Installation

<p>
If the compilation was successful, you should install the software.
Edit the <tt/Makefile/ and set the LIBDIR to the installation
directory. Since, the ES is executed by the mail system, and not by a
user, this directory shouldn't be globally executable.

When satisfied, type <tt/make install/.

Three executables are installed in LIBDIR:
<descrip>
<tag/eti/ The email transport interface. This program receives
incoming mail, identifies the user, and delivers the mail request
to the monitor or kernel (depending on configuration).
<tag/monitor/ The monitor
 is optional component. The main objective
of the monitor is to limit the number of simultanous running kernel
processes.
<tag/kernel/ The kernel process is the core of the ES. It parses
the user's requests and interfaces the Z39.50 protocols.
</descrip>

The <tt/sendmail/ or a similar program delivers the mail to the
<tt/eti/ program. The <tt/sendmail/ program usually runs as user
<tt/mail/ or some other special user name. We strongly suggest that
you create a special user and group for the ES software. In this case
you should use <tt/chmod/ to and set the 'set user ID on execution'
bits on the executable files and give that user read/write/execute
permissions in LIBDIR.

The mail system needs to know about the ES. Pick some name that serves
as the ES user and edit <tt/aliases/ used by your mail system (usually 
<tt>usr/lib/aliases</tt>). Now add the following line:

<tt>es:"|/usr/local/lib/es/eti </tt><em>options</em><tt>"</tt>

In this example the mail user name was <tt/es/ and the LIBDIR was
<tt>/usr/local/lib/es</tt>.

The ES system can operate with or without the monitor. When using
the monitor the number of simultanous running kernels can be
controlled. If the <tt>eti</tt> program is started with 
two dashes (<tt>--</tt>) it will operate without the monitor and
the options specified after the two dashes are transferred to the
kernel.

<sect1>With the monitor

<p>
The monitor must be running at all times in this mode. You should
start the monitor in one of your boot scripts (rc). For example this
might be put in a boot script:

<tscreen><verb>
(cd /usr/local/lib/es; ./monitor -d -l mon.log -- -d -l kernel.log &)
</verb></tscreen>

Here the monitor is started with the options <tt>-d -l mon.log</tt>
and the options after the two dashes are transferred to the 
kernel. In this mode, the eti should contact the monitor (and not
the kernel), so the following might be put in the aliases file:

<tscreen><verb>
es:"|/usr/local/lib/es/eti -c /usr/local/lib/es"
</verb></tscreen>

The eti sets current directory to the path specified by option <tt>-c</tt>.

<sect1>Without the monitor

<p>
In this mode you should never start the monitor. 
The eti will contact the kernel directly. The following line could
be put in your aliases file:

<tscreen><verb>
es:"|/usr/local/lib/es/eti -c /usr/local/lib/es -- -d -l kernel.log"
</verb></tscreen>

<sect1>eti

<p>
The eti program accepts the following options:
<descrip>
<tag><tt>-l </tt>log</tag> The log file. If absent stderr is used.
<tag><tt>-d</tt></tag> Turns on debugging.
<tag><tt>-c </tt>dir</tag> Sets current directory to dir.
<tag><tt>-H</tt></tag> Help message.
<tag><tt>--</tt></tag> Indicates that the eti program should contact the
kernel (and not the monitor. All options after this one are transferred
to the kernel
</descrip>

<sect1>monitor

<p>
The monitor program accepts the following command line options:
<descrip>
<tag><tt>-l </tt>log</tag> The log file. If absent stderr is used.
<tag><tt>-d</tt></tag> Turns on debugging.
<tag><tt>-H</tt></tag> Help message.
<tag><tt>--</tt></tag> Precedes options that are transferred to the kernel
</descrip>

The monitor normally reads the resource <tt>default.res</tt> in
current directory. You can change this behaviour by specifying an
alternate file on the command line.

<sect1>kernel

<p>
List of options observed by the kernel:
<descrip>
<tag><tt>-d</tt></tag> Turns on debugging.
<tag><tt>-t </tt>target</tag> Opens connection to target (for testing only).
<tag><tt>-g </tt>lang</tag> Set language name.
<tag><tt>-o </tt>res</tag> Overriding resource file name. These
resources override both <tt>default.res</tt> and all user resources.
<tag><tt>-h </tt>host</tag> Override host name (for testing only).
<tag><tt>-p </tt>port</tag> Override port no (for testing only).
<tag><tt>-l </tt>log</tag> Specify log file.
<tag><tt>-H</tt></tag> Help message.
</descrip>

The kernel normally reads the resource <tt>default.res</tt> in
current directory. You can change this behaviour by specifying an
alternate file on the command line.

<sect>Managing the system

<sect1>Summary of files
<p>

To maintain the ES you need to know the files it uses. These are:
<descrip>
<tag>*.res</tag> Resource files with several settings that control
 how the system operates, such as definition of targets, messages, etc.
<tag>*.bib</tag> Bib-1 attribute mapping files. These files describe
 the mapping between CCL and the RPN query.
<tag>user.db</tag> Database of users. Only the eti process accesses
 this file.
<tag>user.*.r</tag> Resource file for a user &mdash; accessed by the kernel
 &mdash; only created when the user uses the <tt>def</tt> command.
<tag>user.*.p</tag> Persistency file for a user &mdash; accessed by
 the kernel
 process. 
</descrip>

The ES system is mostly managed by resource files. The following
are example resource files that comes with the ES:
<descrip>
<tag><tt>default.res</tt></tag> General resource with reasonable defaults. 
 This file is read by the monitor and the kernel.
<tag><tt>loc.res</tt></tag> Resource file for Library of Congress test
 server.
<tag><tt>drewdb.res</tt></tag> Resource file for Data Research's test
 server.
<tag><tt>lang.uk.res</tt></tag> Resource file for english conversation.
<tag><tt>lang.dk.res</tt></tag> Resource file for danish conversation.
</descrip>

<sect1>Resources

<p>
Most general resources should be set in the file <tt>default.res</tt>. 
Some of the resources may be changed (overridden) by the user, while 
others may be overridden by individual target defintions. 
The complete scenario is depicted below:
<tscreen><verb>
       +-------------+
       | default.res |
       +-------------+
              |          +--------------+
              |<---------| "target.res" | 
              |          +--------------+
              |
              |          +--------------+
              |<---------| user.x.res   |
              |          +--------------+
              |
              |          +--------------+
              |<---------| "lang.res"   |
              |          +--------------+
              |
              |          +--------------+
              |<---------| "override"   |
              |          +--------------+
            result
</verb></tscreen>

The following describes the general resources:
<descrip>
<tag>gw.reply.mta</tag> Name of MTA program &mdash; default
 <tt>/usr/lib/sendmail</tt>.
<tag>gw.reply.tmp.prefix</tag> Prefix of temporary files used by the ES.
<tag>gw.reply.tmp.dir</tag> Name of directory with temporary files.
<tag>gw.marc.log</tag> If this resource is specified, retrieved MARC
 records will be appended to this file.
<tag>gw.timeout</tag> Idle time before the kernel exits. When the
 kernel exits, the Z39.50 persistency layer will reconnect when
 necessary.
<tag>gw.resultset</tag> If this setting is 1, the Z39.50 client will
 use named result sets. If 0, the Z39.50 system will always use 
 <tt/Default/ as result-set name.
<tag>gw.persist</tag> If this setting 1, the persistency is enabled;
 disabled otherwise.
<tag>gw.max.process</tag> This settings is the maximum number of
 simultaneous kernel processes &mdash; only used by the monitor.
<tag>gw.ignore.which</tag> Some targets doesn't indicate whether
 a record is a diagnostic messaage or a database record. If this
 setting is 1, the ES will always try to interpret the record as a
 database record in ISO2709 format. If 0, the ES will use the
 record type.
<tag>gw.default.show</tag> Default number of records to retrieve and display
 when using the show command. This setting may be changed by the user
 with the <tt>def defaultshow</tt> command.
<tag>gw.max.show</tag> This setting specifies the maximum number of
 records the user may retrieve in one show command &mdash; default 100.
<tag>gw.autoshow</tag> Number of records to retrieve in a find
 command &mdash; default 0. This setting may be changed by the user by
 the <tt>def autoshow</tt> command.
<tag>gw.display.format</tag> Default display format. This setting may
 be changed by the user by the <tt>def f</tt> command.
<tag>gw.language</tag> Current language. This setting may be
changed by the user with the <tt>def lang</tt> command. When the 
 langauge is set to something, say x, then the resource gw.lang.x 
 should hold a name of a resource file read by the kernel.
<tag>gw.lang.<em/x/</tag> Specifies name of resource file for 
 language <em/x/.
<tag>gw.target.<em/name/ </tag> Name of resource file of target
 <em/name/.
<tag>gw.portno</tag> Z39.50 target port number &mdash; default 210.
<tag>gw.hostname</tag> Z39.50 target host name.
<tag>gw.bibset</tag> Name of file with Bib-1 attribute mapping.
<tag>gw.databases</tag> Available databases on target.
<tag>gw.description</tag> Description of a target. This message
 is returned to the user when the connection is established with the
 target.
<tag>gw.account</tag> Z39.50 Authentication string &mdash; default
 empty (i.e. none).
</descrip>

<sect1>Messages

<p>
There are several resource settings that deal with language 
dependencies. These fall into the following categories that
depend on the resource name prefixes:
<descrip>
<tag>gw.msg</tag> Miscellaneous messages.
<tag>gw.err</tag> Error messages.
<tag>gw.bib1.diag.<em/no/</tag> Diagnostic error message indicated by
 <em/no/.
<tag>gw.help</tag> Help/description of various commands.
<tag>ccl.command</tag> CCL command names.
<tag>ccl.token</tag> CCL tokens names.
</descrip>

Refer to the sample files, <tt>default.res</tt>, <tt>lang.uk.res</tt>
and <tt>lang.dk.res</tt> for all available settings.

<sect1>Target definitions

<p>
To add a target definition called <em/mytarget/ you need to make a resource 
entry in <tt>default.res</tt> called <tt>gw.target.</tt><em>mytarget</em>.
The value of this resource is the name of a resource file &mdash; for 
example <em>mytarget</em><tt>.res</tt>. The resource file should at least
define the resources: <tt/gw.hostname/, <tt/gw.databases/ and 
<tt/gw.description/. You might also consider specifying
<tt/gw.account/, <tt/gw.bibset/, <tt/gw.resultset/ and <tt/gw.portno/
in the target resource file. The user only needs to use the command
<tt>target </tt><em>mytarget</em> to use the target. Also, since we
already specified database names, the user doesn't need to use the
<tt/base/ command.

<sect1>CCL to RPN mapping

<p>
The mapping between CCL-queries and RPN are stored in files &mdash;
normally with the suffix <tt>.bib</tt>. We will refer these
files as bibset-files. You might consult the file <tt/default.bib/
to see an example of such file.

The mapping is necessary because targets usually only support a little
subset of the Bib-1 attribute set and because the CCL qualifiers
(field names) are not standardized. A bibset-file is specified
by the <tt/gw.bibset/ resource.

Column zero of a bib-file line either hold a hash character (<tt/#/)
indicating a comment in which case the rest of the line is
ignored; or a CCL qualifier.

The name of the CCL qualifier is up to you. However, the special
qualifier name <tt/term/ applies to the case where no qualifier
is specified in CCL. The CCL qualifier is
followed by one or more mapping specifications. A mapping
specification takes the form:

<em/type/<tt/=/<em/value/<tt/,/<em/value/...

The type is simply one of the six Bib-1 attribute query types:
<descrip>
<tag/u/ Use attribute. Value is an integer.
<tag/t/ Truncation attribute. Value is an integer; or the
 value is a combination of:
  <descrip>
  <tag/l/ This character indicates that the CCL parser should allow
  left truncation (2) if indicated by a <tt/?/ on the left side
  of a term.
  <tag/r/ This character indicates that the CCL parser should allow
  right truncation (1) if indicated by a <tt/?/ on the right side
  of a term.
  <tag/b/ This character indicates that the CCL parser should allow
  both left and right (3) truncation indicated by a <tt/?/ on both
  left and right side of a term.
  <tag/n/ This character indicates that the CCL parser should announce
  no truncation (100) if no truncation was indicated.
  </descrip>
<tag/p/ Position attribute. Valus is an integer.
<tag/s/ Structure attribute. Value is an integer; or the
 value is <tt/pw/ in which case the CCL parser announces word (2) or
 phrase (1) depending on the number of adjacent terms.
<tag/r/ Relation attribute. Value is an integer; or the value is
 <tt/o/ in which case, the CCL parser will select <em/less than/, 
 <em/less than or equal/, ... <em/greater than/ &mdash; depending on
 the relation specified in CCL.
<tag/p/ Position attribute. Value is an integer.
</descrip>

Consider these bibset-lines:
<tscreen><verb>
term t=l,r,b s=pw
au= u=1 t=l,r,b s=pw
date u=30 r=o
</verb></tscreen>
The first line describes the mapping in when no qualifiers are
present, as in:
<tscreen><verb>
find foo bar?
</verb></tscreen>
In this case the right truncation is enabled and the structure is
<em/phrase/.

The second line is used in this search:
<tscreen><verb>
find au=andersen
</verb></tscreen>
where the use attribute is <em/author/ and the structure is <em/word/.

The third line is used in:
<tscreen><verb>
find date>1990
</verb></tscreen>
where the use attribute is <em/date/ and the relation is <em/greater than/.

<sect>LICENSE

<p>
 Copyright &copy; 1995, the EUROPAGATE consortium (see below).

 The EUROPAGATE consortium members are:

<itemize>
<item>University College Dublin
<item>Danmarks Teknologiske Videnscenter
<item>An Chomhairle Leabharlanna
<item>Consejo Superior de Investigaciones Cientificas
</itemize>

 Permission to use, copy, modify, distribute, and sell this software and
 its documentation, in whole or in part, for any purpose, is hereby granted,
 provided that:

 1. This copyright and permission notice appear in all copies of the
 software and its documentation. Notices of copyright or attribution
 which appear at the beginning of any file must remain unchanged.

 2. The names of EUROPAGATE or the project partners may not be used to
 endorse or promote products derived from this software without specific
 prior written permission.

 3. Users of this software (implementors and gateway operators) agree to
 inform the EUROPAGATE consortium of their use of the software. This
 information will be used to evaluate the EUROPAGATE project and the
 software, and to plan further developments. The consortium may use
 the information in later publications.
 
 4. Users of this software agree to make their best efforts, when
 documenting their use of the software, to acknowledge the EUROPAGATE
 consortium, and the role played by the software in their work.

 THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND,
 EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY
 WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
 IN NO EVENT SHALL THE EUROPAGATE CONSORTIUM OR ITS MEMBERS BE LIABLE
 FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF
 ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA
 OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND
 ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE
 USE OR PERFORMANCE OF THIS SOFTWARE.

</article>