+<sect>Implementation
+
+<p>
+The implementation of the email server includes all the modules described
+in the design deliverable.
+
+The work was roughly carried out as follows:
+<enum>
+<item>The logging facilities and resource management utilities were
+ implemented — virtually all other modules depend on these
+ modules.
+<item>A minimal ES was implemented — including a high-level
+ API to the Z39.50 sub-system and a CCL parser with a few
+ commands, such as FIND and SHOW. This version displayed MARC
+ records in a raw format. This version served as base for the URP.
+<item>The first version of the MARC display formatting tool, FML,
+ was implemented and included in the ES.
+<item>The ETI program was implemented along with the IPC
+ (interprocess communication) utilities based on FIFOs. Facilities
+ to keep connections alive (to Z39.50 targets) was implemented.
+ To identify a user, a file-resident symbol table (small database) was
+ implemented which maps a email username to a unique integer (email userid).
+<item>The protocol persistency was implemented and more CCL commands
+ were added.
+<item>The monitor program was implemented.
+</enum>
+
+The following sections cover the most important modules in the ES and
+deviations from the design.
+
+<sect1>Z39.50 Interface layer
+
+<p>
+The design report specified that the Zdist toolkit from CNIDR would
+be used in the ES to provide access to the Z39.50 protocol. The package
+was choosen bacause it is easy to use and, more important, we felt
+that the API would be reasonably stable and supported.
+
+Nevertheless it turned out that CNIDR choose to change the API
+completely around January 1995 and announced a new version
+called zdist102b1-1.
+
+<em>Note: As of this date the newest version of Zdist is still
+zdist102b1-1. CNIDR seems to concentracte on their Isite package
+which also includes a Zdist package presumably similar to the
+standalone Zdist package</em>
+
+During the work with the Zdist package a few bugs were discovered.
+Fortunately, they could be solved within a few days. We also
+discovered that the package lacks result-set references.
+We posted the bug fixes to Kevin Gamiel who is responsible for
+the package but we didn't get responses. So, eventually, we weren't
+satisfied with the package after all.
+
+In February some of us began the development of a new Z39.50 package
+called YAZ — in retrospect somewhat motivated by the
+experiences with existing Z39.50/SR toolkits.
+
+To support result-set references we chose to incorporate a YAZ
+interface in the ES also. And we designed and implemented a
+simple high-level Z39.50 origin API that supported both Zdist and YAZ.
+
+The protocol persistency module was implemented on top of
+the high-level API and not on top of Zdist. The obvious
+advantage is that the persistency module is not tied to one
+particular Z39.50/SR package.
+
+Persistency information stored for each user is simply:
+<itemize>
+<item>hostname and port number.
+<item>authentication string
+<item>selected database(s)
+<item>next result set number
+<item>next result set position
+<item>result set information
+</itemize>
+
+Information about each result set includes:
+<itemize>
+<item>name
+<item>size (number of hits)
+<item>database(s)
+<item>query
+</itemize>
+
+A persistency file is removed each time a new target is selected.
+It is our experiences that the persistency files are very small.
+
+<sect1>CCL
+
+<p>
+The CCL was implemented as described in the design. A CCL utility
+was made as a separate module which implements a tokenization
+package and a parser which translates from FIND to RPN. The
+data structure used to represent the RPN query is also used in
+Z39.50 search API on top of YAZ or Zdist.
+
+The CCL parser is quite configurable. Token names can be redefined to
+one or more names (aliases). Also, the specification of mapping
+between CCL field names (qualifiers) and Bib-1 attributes can be
+specified in either the C API or a file.
+
+Although the Z39.50 system in the ES uses the Bib-1 attribute set, the
+CCL parser itself is not tied to Bib-1.
+
+<sect1>FML
+
+<p>
+The FML system is used to handle the presentation of MARC
+records. There are some deviations to the design report, however.
+The most important changes are:
+<itemize>
+<item>The <tt/expr/ function is not implemented. Instead arithmetic
+operators <tt/plus/, <tt/minus/, <tt/mult/ and <tt/div/ are
+implemented. Also relational operators <tt/gt/, <tt/lt/ ... are
+implemented.
+<item>The <tt/lindex/ function is called <tt/index/ and it is a binary
+operator where the left operand is the list and the right operand is
+the index integer.
+<item>The MARC extraction routines are not implemented.
+Instead, a MARC record is transferred as an argument
+to a formatting-routine (in list notation). The formatting
+routine then extracts fields from the list by list/string
+manipulation functions.
+<item>A new statement, <tt/bin/, is implemented to define
+binary operators (functions).
+</itemize>
+
+<sect1>IPC
+
+<p>
+As described in the design, FIFOs are used to communicate between
+the ETI, monitor and kernel. The ES can run without the monitor,
+however. The primary reason for the presence of the monitor was
+to assure that the kernel releases the resources used by the
+persistency layer. But, since the persistency layer did turn out to
+use virtually no disk space at all, there was no point in starting
+a kernel process to remove its files — hence this facility
+was not implemented. The only purpose of the monitor is to keep the
+number of running kernels at a maximum level and even that
+is probably useless since most unices will swap kernel processes
+out anyway.
+
+The idle time
+before a kernel exits and saves its persistency file is not
+controlled by the monitor. Saving the persistency file and
+keeping it is usually a good approach — even when a
+user doesn't reference/show old result-sets since the user
+has a notion of <em/current target/ and database.
+
+<sect1>Source
+
+<p>
+In this section a short description of each source module is
+given. Each module is implemented in a separate sub directory.
+Any public headers are located in the <tt/include/ directory.
+
+<descrip>
+<tag/res+log/ is an implementation of the logging system
+and the resource management sub system. Note that the
+resource module depends on the logging facility. Logging
+is implemented in <tt>gw-log.c</tt> and <tt/gw-log.h/. The
+file <tt>gw-log-test.c</tt> is small test program for the
+logging system. The core of the resource management is implemented
+in <tt>gw-res.c</tt>. The files <tt>gw-res-bool.c</tt> and
+<tt>gw-res-int.c</tt> implement two utility routines &mdash
+on top of the resource management. The header file
+<tt>gw-resp.h</tt> is a private header file and <tt>gw-res.h</tt>
+is a public header file.
+
+<tag/ccl/ implements CCL to RPN mapping and a tokenization
+ utility for other CCL commands. The mapping function is
+ implemented in <tt>cclfind.c</tt>. Qualifiers are handled in
+ <tt>cclqual.c</tt> while reading of qualifier mappings from a
+ file is implemented in <tt>cclqfile.c</tt>. Scanning is implemented
+ in <tt>ccltoken.c</tt>. String utilities, which might be changed if
+ other character sets are needed, is implemented in
+ <tt>cclstr.c</tt>. Table of error messages is implemented in
+ <tt>cclerrms.c</tt>.
+
+<tag/util/ implements various utilities:
+ <descrip>
+ <tag>MARC utility</tag> implemented in <tt>iso2709</tt>...
+ <tag>Database utility</tag> implemented in <tt>gw-db.[ch]</tt>. This
+ utility is used to map a user (email) to an integer.
+ <tag>String queue utility</tag> implemented in <tt>strqueue.[ch]</tt>. This
+ utiltiy is used to queue incoming mail in the ETI, kernel and
+ the monitor.
+ <tag>Pretty printer</tag> implemented in <tt>ttyemit.[ch]</tt>
+ — used by the URP.
+ <tag>FIFO IPC utiltiy</tag> implemented in <tt>gip*.[ch]</tt> —
+ used by the ETI, kernel and monitor.
+ </descrip>
+
+<tag/fml/ implements FML. The top level functions are implemented
+ in <tt>fml.c</tt>, <tt>fmlcall.c</tt> and <tt>fmlcalls.c</tt>.
+ Scanning is implemented in <tt>fmltoken.c</tt>.
+ Memory management is implemented in <tt>fmlmem.c</tt>.
+ Arithmetic operators are implemented in <tt>fmlarit.c</tt>.
+ String manipulation functions are implemented in <tt>fmlstr.c</tt>.
+ Relational operators are implemented in <tt>fmlrel.c</tt>.
+ List maniuplations are performed in <tt>fmllist.c</tt>.
+ FML symbol table management is implemented in <tt>fmlsym.c</tt>.
+ Conversion from ISO2709 to list notation is implemented in
+ <tt>fmlmarc.c</tt>.
+
+<tag/zlayer-zdist/ implements the high-level Z39.50 API on top
+ of Zdist. This task is implemented in <tt>zaccess.c</tt>. The
+ public header file is called <tt>zaccess.h</tt>.
+
+<tag/zlayer-yaz/ implements the high-level Z39.50 API on top
+ of YAZ. This task is implemented in <tt>zaccess.c</tt>. The
+ public header file is called <tt>zaccess.h</tt>.
+
+<tag/kernel/ implements the ETI, kernel and monitor. The kernel
+ itself is implemented in <tt>main.c</tt>, <tt>urp.c</tt> and
+ <tt>persist.c</tt>. The ETI is implemented in <tt>eti.c</tt> and
+ the monitor is implemented <tt>monitor.c</tt>.
+</descrip>
+
projects / egate.git / blobdiff