+<sect2>Close
+
+<p>
+If a Close PDU is received, the server will respond with a Close PDU
+with reason=FINISHED, no matter which protocol version was negotiated
+during initialization. If the protocol version is 3 or more, the
+server will generate a Close PDU under certain circumstances,
+including a session timeout (60 minutes by default), and certain kinds of
+protocol errors. Once a Close PDU has been sent, the protocol
+association is considered broken, and the transport connection will be
+closed immediately upon receipt of further data, or following a short
+timeout.
+
+<sect>The Record Model
+
+<p>
+The Zebra system is designed to support a wide range of data management
+applications. The system can be configured to handle virtually any
+kind of structured data. Each record in the system is associated with
+a <it/record schema/ which lends context to the data elements of the
+record. Any number of record schema can coexist in the system.
+Although it may be wise to use only a single schema within
+one database, the system poses no such restrictions.
+
+The record model described in this chapter applies to the fundamental
+record type <tt>grs</tt> as introduced in
+section <ref id="record-types" name="Record Types">.
+
+Records pass through three different states during processing in the
+system.
+
+<itemize>
+<item>When records are accessed by the system, they are represented
+in their local, or native format. This might be SGML or HTML files,
+News or Mail archives, MARC records. If the system doesn't already
+know how to read the type of data you need to store, you can set up an
+input filter by preparing conversion rules based on regular
+expressions and a flexible scripting language (Tcl). The input filter
+produces as output an internal representation:
+
+<item>When records are processed by the system, they are represented
+in a tree-structure, constructed by tagged data elements hanging off a
+root node. The tagged elements may contain data or yet more tagged
+elements in a recursive structure. The system performs various
+actions on this tree structure (indexing, element selection, schema
+mapping, etc.),
+
+<item>Before transmitting records to the client, they are first
+converted from the internal structure to a form suitable for exchange
+over the network - according to the Z39.50 standard.
+</itemize>
+
+<sect1>Local Representation<label id="local-representation">
+
+<p>
+As mentioned earlier, Zebra places few restrictions on the type of
+data that you can index and manage. Generally, whatever the form of
+the data, it is parsed by an input filter specific to that format, and
+turned into an internal structure that Zebra knows how to handle. This
+process takes place whenever the record is accessed - for indexing and
+retrieval.
+
+<sect2>Canonical Input Format
+
+<p>
+Although input data can take any form, it is sometimes useful to
+describe the record processing capabilities of the system in terms of
+a single, canonical input format that gives access to the full
+spectrum of structure and flexibility in the system. In Zebra, this
+canonical format is an &dquot;SGML-like&dquot; syntax.
+
+To use the canonical format specify <tt>grs.sgml</tt> as the record
+type,
+
+Consider a record describing an information resource (such a record is
+sometimes known as a <it/locator record/). It might contain a field
+describing the distributor of the information resource, which might in
+turn be partitioned into various fields providing details about the
+distributor, like this:
+
+<tscreen><verb>
+<Distributor>
+ <Name> USGS/WRD &etago;Name>
+ <Organization> USGS/WRD &etago;Organization>
+ <Street-Address>
+ U.S. GEOLOGICAL SURVEY, 505 MARQUETTE, NW
+ &etago;Street-Address>
+ <City> ALBUQUERQUE &etago;City>
+ <State> NM &etago;State>
+ <Zip-Code> 87102 &etago;Zip-Code>
+ <Country> USA &etago;Country>
+ <Telephone> (505) 766-5560 &etago;Telephone>
+&etago;Distributor>
+</verb></tscreen>
+
+<it>NOTE: The indentation used above is used to illustrate how Zebra
+interprets the markup. The indentation, in itself, has no
+significance to the parser for the canonical input format, which
+discards superfluous whitespace.</it>
+
+The keywords surrounded by <...> are <it/tags/, while the
+sections of text in between are the <it/data elements/. A data element
+is characterized by its location in the tree that is made up by the
+nested elements. Each element is terminated by a closing tag -
+beginning with &etago;, and containing the same symbolic tag-name as
+the corresponding opening tag. The general closing tag - &etago;> -
+terminates the element started by the last opening tag. The
+structuring of elements is significant. The element <bf/Telephone/,
+for instance, may be indexed and presented to the client differently,
+depending on whether it appears inside the <bf/Distributor/ element,
+or some other, structured data element such a <bf/Supplier/ element.
+
+<sect3>Record Root
+
+<p>
+The first tag in a record describes the root node of the tree that
+makes up the total record. In the canonical input format, the root tag
+should contain the name of the schema that lends context to the
+elements of the record (see section <ref id="internal-representation"
+name="Internal Representation">). The following is a GILS record that
+contains only a single element (strictly speaking, that makes it an
+illegal GILS record, since the GILS profile includes several mandatory
+elements - Zebra does not validate the contents of a record against
+the Z39.50 profile, however - it merely attempts to match up elements
+of a local representation with the given schema):
+
+<tscreen><verb>
+<gils>
+ <title>Zen and the Art of Motorcycle Maintenance&etago;title>
+&etago;gils>
+</verb></tscreen>
+
+<sect3>Variants
+
+<p>
+Zebra allows you to provide individual data elements in a number of
+<it/variant forms/. Examples of variant forms are textual data
+elements which might appear in different languages, and images which
+may appear in different formats or layouts. The variant system in
+Zebra is
+essentially a representation of the variant mechanism of
+Z39.50-1995.
+
+The following is an example of a title element which occurs in two
+different languages.
+
+<tscreen><verb>
+<title>
+ <var lang lang "eng">
+ Zen and the Art of Motorcycle Maintenance&etago;>
+ <var lang lang "dan">
+ Zen og Kunsten at Vedligeholde en Motorcykel&etago;>
+&etago;title>
+</verb></tscreen>
+
+The syntax of the <it/variant element/ is <tt><<bf/var/ <it/class
+type value/></tt>. The available values for the <it/class/ and
+<it/type/ fields are given by the variant set that is associated with the
+current schema (see section <ref id="variant-set" name="Variant Set
+File">).
+
+Variant elements are terminated by the general end-tag &etago;>, by
+the variant end-tag &etago;var>, by the appearance of another variant
+tag with the same <it/class/ and <it/value/ settings, or by the
+appearance of another, normal tag. In other words, the end-tags for
+the variants used in the example above could have been saved.
+
+Variant elements can be nested. The element
+
+<tscreen><verb>
+<title>
+ <var lang lang "eng"><var body iana "text/plain">
+ Zen and the Art of Motorcycle Maintenance
+&etago;title>
+</verb></tscreen>
+
+Associates two variant components to the variant list for the title
+element.
+
+Given the nesting rules described above, we could write
+
+<tscreen><verb>
+<title>
+ <var body iana "text/plain>
+ <var lang lang "eng">
+ Zen and the Art of Motorcycle Maintenance
+ <var lang lang "dan">
+ Zen og Kunsten at Vedligeholde en Motorcykel
+&etago;title>
+</verb></tscreen>
+
+The title element above comes in two variants. Both have the IANA body
+type &dquot;text/plain&dquot;, but one is in English, and the other in
+Danish. The client, using the element selection mechanism of Z39.50,
+can retrieve information about the available variant forms of data
+elements, or it can select specific variants based on the requirements
+of the end-user.
+
+<sect2>Input Filters
+
+<p>
+In order to handle general input formats, Zebra allows the
+operator to define filters which read individual records in their native format
+and produce an internal representation that the system can
+work with.
+
+Input filters are ASCII files, generally with the suffix <tt/.flt/.
+The system looks for the files in the directories given in the
+<bf/profilePath/ setting in the <tt/zebra.cfg/ files. The record type
+for the filter is <tt>grs.regx.</tt><it>filter-filename</it>
+(fundamental type <tt>grs</tt>, file read type <tt>regx</tt>, argument
+<it>filter-filename</it>).
+
+Generally, an input filter consists of a sequence of rules, where each
+rule consists of a sequence of expressions, followed by an action. The
+expressions are evaluated against the contents of the input record,
+and the actions normally contribute to the generation of an internal
+representation of the record.
+
+An expression can be either of the following:
+
+<descrip>
+<tag/INIT/The action associated with this expression is evaluated
+exactly once in the lifetime of the application, before any records
+are read. It can be used in conjunction with an action that
+initializes tables or other resources that are used in the processing
+of input records.
+
+<tag/BEGIN/Matches the beginning of the record. It can be used to
+initialize variables, etc. Typically, the <bf/BEGIN/ rule is also used
+to establish the root node of the record.
+
+<tag/END/Matches the end of the record - when all of the contents
+of the record has been processed.
+
+<tag>/pattern/</tag>Matches a string of characters from the input
+record.
+
+<tag/BODY/This keyword may only be used between two patterns. It
+matches everything between (not including) those patterns.
+
+<tag/FINISH/THe expression asssociated with this pattern is evaluated
+once, before the application terminates. It can be used to release
+system resources - typically ones allocated in the <bf/INIT/ step.
+
+</descrip>
+
+An action is surrounded by curly braces ({...}), and consists of a
+sequence of statements. Statements may be separated by newlines or
+semicolons (;). Within actions, the strings that matched the
+expressions immediately preceding the action can be referred to as
+$0, $1, $2, etc.
+
+The available statements are:
+
+<descrip>
+
+<tag>begin <it/type [parameter ... ]/</tag>Begin a new
+data element. The type is one of the following:
+<descrip>
+<tag/record/Begin a new record. The followingparameter should be the
+name of the schema that describes the structure of the record, eg.
+<tt/gils/ or <tt/wais/ (see below). The <tt/begin record/ call should
+precede
+any other use of the <bf/begin/ statement.
+
+<tag/element/Begin a new tagged element. The parameter is the
+name of the tag. If the tag is not matched anywhere in the tagsets
+referenced by the current schema, it is treated as a local string
+tag.
+
+<tag/variant/Begin a new node in a variant tree. The parameters are
+<it/class type value/.
+
+</descrip>
+
+<tag/data/Create a data element. The concatenated arguments make
+up the value of the data element. The option <tt/-text/ signals that
+the layout (whitespace) of the data should be retained for
+transmission. The option <tt/-element/ <it/tag/ wraps the data up in
+the <it/tag/. The use of the <tt/-element/ option is equivalent to
+preceding the command with a <bf/begin element/ command, and following
+it with the <bf/end/ command.
+
+<tag>end <it/[type]/</tag>Close a tagged element. If no parameter is given,
+the last element on the stack is terminated. The first parameter, if
+any, is a type name, similar to the <bf/begin/ statement. For the
+<bf/element/ type, a tag name can be provided to terminate a specific tag.
+
+</descrip>
+
+The following input filter reads a Usenet news file, producing a
+record in the WAIS schema. Note that the body of a news posting is
+separated from the list of headers by a blank line (or rather a
+sequence of two newline characters.
+
+<tscreen><verb>
+BEGIN { begin record wais }
+
+/^From:/ BODY /$/ { data -element name $1 }
+/^Subject:/ BODY /$/ { data -element title $1 }
+/^Date:/ BODY /$/ { data -element lastModified $1 }
+/\n\n/ BODY END {
+ begin element bodyOfDisplay
+ begin variant body iana "text/plain"
+ data -text $1
+ end record
+ }
+</verb></tscreen>
+
+If Zebra is compiled with support for Tcl (Tool Command Language)
+enabled, the statements described above are supplemented with a complete
+scripting environment, including control structures (conditional
+expressions and loop constructs), and powerful string manipulation
+mechanisms for modifying the elements of a record. Tcl is a popular
+scripting environment, with several tutorials available both online
+and in hardcopy.
+
+<it>NOTE: Tcl support is not currently available, but will be
+included with one of the next alpha or beta releases.</it>
+
+<it>NOTE: Variant support is not currently available in the input
+filter, but will be included with one of the next alpha or beta
+releases.</it>
+
+<sect1>Internal Representation<label id="internal-representation">
+
+<p>
+When records are manipulated by the system, they're represented in a
+tree-structure, with data elements at the leaf nodes, and tags or
+variant components at the non-leaf nodes. The root-node identifies the
+schema that lends context to the tagging and structuring of the
+record. Imagine a simple record, consisting of a 'title' element and
+an 'author' element:
+
+<tscreen><verb>
+ TITLE "Zen and the Art of Motorcycle Maintenance"
+ROOT
+ AUTHOR "Robert Pirsig"
+</verb></tscreen>
+
+A slightly more complex record would have the author element consist
+of two elements, a surname and a first name:
+
+<tscreen><verb>
+ TITLE "Zen and the Art of Motorcycle Maintenance"
+ROOT
+ FIRST-NAME "Robert"
+ AUTHOR
+ SURNAME "Pirsig"
+</verb></tscreen>
+
+The root of the record will refer to the record schema that describes
+the structuring of this particular record. The schema defines the
+element tags (TITLE, FIRST-NAME, etc.) that may occur in the record, as
+well as the structuring (SURNAME should appear below AUTHOR, etc.). In
+addition, the schema establishes element set names that are used by
+the client to request a subset of the elements of a given record. The
+schema may also establish rules for converting the record to a
+different schema, by stating, for each element, a mapping to a
+different tag path.
+
+<sect2>Tagged Elements
+
+<p>
+A data element is characterized by its tag, and its position in the
+structure of the record. For instance, while the tag &dquot;telephone
+number&dquot; may be used different places in a record, we may need to
+distinguish between these occurrences, both for searching and
+presentation purposes. For instance, while the phone numbers for the
+&dquot;customer&dquot; and the &dquot;service provider&dquot; are both
+representatives for the same type of resource (a telephone number), it
+is essential that they be kept separate. The record schema provides
+the structure of the record, and names each data element (defined by
+the sequence of tags - the tag path - by which the element can be
+reached from the root of the record).
+
+<sect2>Variants
+
+<p>
+The children of a tag node may be either more tag nodes, a data node
+(possibly accompanied by tag nodes),
+or a tree of variant nodes. The children of variant nodes are either
+more variant nodes or a data node (possibly accompanied by more
+variant nodes). Each leaf node, which is normally a
+data node, corresponds to a <it/variant form/ of the tagged element
+identified by the tag which parents the variant tree. The following
+title element occurs in two different languages:
+
+<tscreen><verb>
+ VARIANT LANG=ENG "War and Peace"
+TITLE
+ VARIANT LANG=DAN "Krig og Fred"
+</verb></tscreen>
+
+Which of the two elements are transmitted to the client by the server
+depends on the specifications provided by the client, if any.
+
+In practice, each variant node is associated with a triple of class,
+type, value, corresponding to the variant mechanism of Z39.50.
+
+<sect2>Data Elements
+
+<p>
+Data nodes have no children (they are always leaf nodes in the record
+tree).
+
+<it>NOTE: Documentation needs extension here about types of nodes - numerical,
+textual, etc., plus the various types of inclusion notes.</it>
+
+<sect1>Configuring Your Data Model<label id="data-model">
+
+<p>
+The following sections describe the configuration files that govern
+the internal management of data records. The system searches for the files
+in the directories specified by the <bf/profilePath/ setting in the
+<tt/zebra.cfg/ file.
+
+<sect2>The Abstract Syntax
+
+<p>
+The abstract syntax definition (also known as an Abstract Record
+Structure, or ARS) is the focal point of the
+record schema description. For a given schema, the ABS file may state any
+or all of the following:
+
+<itemize>
+<item>The object identifier of the Z39.50 schema associated
+with the ARS, so that it can be referred to by the client.
+
+<item>The attribute set (which can possibly be a compound of multiple
+sets) which applies in the profile. This is used when indexing and
+searching the records belonging to the given profile.
+
+<item>The Tag set (again, this can consist of several different sets).
+This is used when reading the records from a file, to recognize the
+different tags, and when transmitting the record to the client -
+mapping the tags to their numerical representation, if they are
+known.
+
+<item>The variant set which is used in the profile. This provides a
+vocabulary for specifying the <it/forms/ of data that appear inside
+the records.
+
+<item>Element set names, which are a shorthand way for the client to
+ask for a subset of the data elements contained in a record. Element
+set names, in the retrieval module, are mapped to <it/element
+specifications/, which contain information equivalent to the
+<it/Espec-1/ syntax of Z39.50.
+
+<item>Map tables, which may specify mappings to <it/other/ database
+profiles, if desired.
+
+<item>Possibly, a set of rules describing the mapping of elements to a
+MARC representation.
+
+<item>A list of element descriptions (this is the actual ARS of the
+schema, in Z39.50 terms), which lists the ways in which the various
+tags can be used and organized hierarchically.
+</itemize>
+
+Several of the entries above simply refer to other files, which
+describe the given objects.
+
+<sect2>The Configuration Files
+
+<p>
+This section describes the syntax and use of the various tables which
+are used by the retrieval module.
+
+The number of different file types may appear daunting at first, but
+each type corresponds fairly clearly to a single aspect of the Z39.50
+retrieval facilities. Further, the average database administrator,
+who is simply reusing an existing profile for which tables already
+exist, shouldn't have to worry too much about the contents of these tables.
+
+Generally, the files are simple ASCII files, which can be maintained
+using any text editor. Blank lines, and lines beginning with a (#) are
+ignored. Any characters on a line followed by a (#) are also ignored.
+All other
+lines contain <it/directives/, which provide some setting or value
+to the system. Generally, settings are characterized by a single
+keyword, identifying the setting, followed by a number of parameters.
+Some settings are repeatable (r), while others may occur only once in a
+file. Some settings are optional (o), whicle others again are
+mandatory (m).
+
+<sect2>The Abstract Syntax (.abs) Files
+
+<p>
+The name of this file type is slightly misleading in Z39.50 terms,
+since, apart from the actual abstract syntax of the profile, it also
+includes most of the other definitions that go into a database
+profile.
+
+When a record in the canonical, SGML-like format is read from a file
+or from the database, the first tag of the file should reference the
+profile that governs the layout of the record. If the first tag of the
+record is, say, <tt><gils></tt>, the system will look for the profile
+definition in the file <tt/gils.abs/. Profile definitions are cached,
+so they only have to be read once during the lifespan of the current
+process.
+
+When writing your own input filters, the <bf/record-begin/ command
+introduces the profile, and should always be called first thing when
+introducing a new record.
+
+The file may contain the following directives:
+
+<descrip>
+<tag>name <it/symbolic-name/</tag> (m) This provides a shorthand name or
+description for the profile. Mostly useful for diagnostic purposes.
+
+<tag>reference <it/OID-name/</tag> (m) The reference name of the OID for
+the profile. The reference names can be found in the <bf/util/
+module of <bf/YAZ/.
+
+<tag>attset <it/filename/</tag> (m) The attribute set that is used for
+indexing and searching records belonging to this profile.
+
+<tag>tagset <it/filename/</tag> (o) The tag set (if any) that describe
+that fields of the records.
+
+<tag>varset <it/filename/</tag> (o) The variant set used in the profile.
+
+<tag>maptab <it/filename/</tag> (o,r) This points to a
+conversion table that might be used if the client asks for the record
+in a different schema from the native one.
+
+<tag>marc <it/filename/</tag> (o) Points to a file containing parameters
+for representing the record contents in the ISO2709 syntax. Read the
+description of the MARC representation facility below.
+
+<tag>esetname <it/name filename/</tag> (o,r) Associates the
+given element set name with an element selection file. If an (@) is
+given in place of the filename, this corresponds to a null mapping for
+the given element set name.
+
+<tag>any <it/tags/</tag> (o) This directive specifies a list of
+attributes which should be appended to the attribute list given for each
+element. The effect is to make every single element in the abstract
+syntax searchable by way of the given attributes. This directive
+provides an efficient way of supporting free-text searching across all
+elements. However, it does increase the size of the index
+significantly. The attributes can be qualified with a structure, as in
+the <bf/elm/ directive below.
+
+<tag>elm <it/path name attributes/</tag> (o,r) Adds an element
+to the abstract record syntax of the schema. The <it/path/ follows the
+syntax which is suggested by the Z39.50 document - that is, a sequence
+of tags separated by slashes (/). Each tag is given as a
+comma-separated pair of tag type and -value surrounded by parenthesis.
+The <it/name/ is the name of the element, and the <it/attributes/
+specifies which attributes to use when indexing the element in a
+comma-separated list. A ! in
+place of the attribute name is equivalent to specifying an attribute
+name identical to the element name. A - in place of the attribute name
+specifies that no indexing is to take place for the given element. The
+attributes can be qualified with a &dquot;p&dquot; or &dquot;w&dquot;
+to specify either word or phrase (complete field) indexing.
+</descrip>
+
+<it>
+NOTE: The mechanism for controlling indexing is not adequate for
+complex databases, and will probably be moved into a separate
+configuration table eventually.
+</it>
+
+The following is an excerpt from the abstract syntax file for the GILS
+profile.
+
+<tscreen><verb>
+name gils
+reference GILS-schema
+attset gils.att
+tagset gils.tag
+varset var1.var
+
+maptab gils-usmarc.map
+
+# Element set names
+
+esetname VARIANT gils-variant.est # for WAIS-compliance
+esetname B gils-b.est
+esetname G gils-g.est
+esetname F @
+
+elm (1,10) rank -
+elm (1,12) url -
+elm (1,14) localControlNumber Local-number
+elm (1,16) dateOfLastModification Date/time-last-modified
+elm (2,1) Title !
+elm (4,1) controlIdentifier Identifier-standard
+elm (2,6) abstract Abstract
+elm (4,51) purpose !
+elm (4,52) originator -
+elm (4,53) accessConstraints !
+elm (4,54) useConstraints !
+elm (4,70) availability -
+elm (4,70)/(4,90) distributor -
+elm (4,70)/(4,90)/(2,7) distributorName !
+elm (4,70)/(4,90)/(2,10 distributorOrganization !
+elm (4,70)/(4,90)/(4,2) distributorStreetAddress !
+elm (4,70)/(4,90)/(4,3) distributorCity !
+</verb></tscreen>
+
+<sect2>The Attribute Set (.att) Files<label id="attset-files">
+
+<p>
+This file type describes the <bf/Use/ elements of an attribute set.
+It contains the following directives.
+
+<descrip>
+
+<tag>name <it/symbolic-name/</tag> (m) This provides a shorthand name or
+description for the attribute set. Mostly useful for diagnostic purposes.
+
+<tag>reference <it/OID-name/</tag> (m) The reference name of the OID for
+the attribute set. The reference names can be found in the <bf/util/
+module of <bf/YAZ/.
+
+<tag>ordinal <it/integer/</tag> (m) This value will be used to represent the
+attribute set in the index. Care should be taken that each attribute
+set has a unique ordinal value.
+
+<tag>include <it/filename/</tag> (o,r) This directive is used to
+include another attribute set as a part of the current one. This is
+used when a new attribute set is defined as an extension to another
+set. For instance, many new attribute sets are defined as extensions
+to the <bf/bib-1/ set. This is an important feature of the retrieval
+system of Z39.50, as it ensures the highest possible level of
+interoperability, as those access points of your database which are
+derived from the external set (say, bib-1) can be used even by clients
+who are unaware of the new set.
+
+<tag>att <it/att-value att-name [local-value]/</tag> (o,r) This
+repeatable directive introduces a new attribute to the set. The
+attribute value is stored in the index (unless a <it/local-value/ is
+given, in which case this is stored). The name is used to refer to the
+attribute from the <it/abstract syntax/. </descrip>
+
+This is an excerpt from the GILS attribute set definition. Notice how
+the file describing the <it/bib-1/ attribute set is referenced.
+
+<tscreen><verb>
+name gils
+reference GILS-attset
+include bib1.att
+ordinal 2
+
+att 2001 distributorName
+att 2002 indexTermsControlled
+att 2003 purpose
+att 2004 accessConstraints
+att 2005 useConstraints
+</verb></tscreen>
+
+<sect2>The Tag Set (.tag) Files
+
+<p>
+This file type defines the tagset of the profile, possibly by
+referencing other tag sets (most tag sets, for instance, will include
+tagsetG and tagsetM from the Z39.50 specification. The file may
+contain the following directives.
+
+<descrip>
+<tag>name <it/symbolic-name/</tag> (m) This provides a shorthand name or
+description for the tag set. Mostly useful for diagnostic purposes.
+
+<tag>reference <it/OID-name/</tag> (o) The reference name of the OID for
+the tag set. The reference names can be found in the <bf/util/
+module of <bf/YAZ/. The directive is optional, since not all tag sets
+are registered outside of their schema.
+
+<tag>type <it/integer/</tag> (m) The type number of the tag within the schema
+profile.
+
+<tag>include <it/filename/</tag> (o,r) This directive is used
+to include the definitions of other tag sets into the current one.
+
+<tag>tag <it/number names type/</tag> (o,r) Introduces a new
+tag to the set. The <it/number/ is the tag number as used in the protocol
+(there is currently no mechanism for specifying string tags at this
+point, but this would be quick work to add). The <it/names/ parameter
+is a list of names by which the tag should be recognized in the input
+file format. The names should be separated by slashes (/). The
+<it/type/ is th recommended datatype of the tag. It should be one of
+the following:
+<itemize>
+<item>structured
+<item>string
+<item>numeric
+<item>bool
+<item>oid
+<item>generalizedtime
+<item>intunit
+<item>int
+<item>octetstring
+<item>null
+</itemize>
+</descrip>
+
+The following is an excerpt from the TagsetG definition file.
+
+<tscreen><verb>
+name tagsetg
+reference TagsetG
+type 2
+
+tag 1 title string
+tag 2 author string
+tag 3 publicationPlace string
+tag 4 publicationDate string
+tag 5 documentId string
+tag 6 abstract string
+tag 7 name string
+tag 8 date generalizedtime
+tag 9 bodyOfDisplay string
+tag 10 organization string
+</verb></tscreen>
+
+<sect2>The Variant Set (.var) Files<label id="variant-set">
+
+<p>
+The variant set file is a straightforward representation of the
+variant set definitions associated with the protocol. At present, only
+the <it/Variant-1/ set is known.
+
+These are the directives allowed in the file.
+
+<descrip>
+<tag>name <it/symbolic-name/</tag> (m) This provides a shorthand name or
+description for the variant set. Mostly useful for diagnostic purposes.
+
+<tag>reference <it/OID-name/</tag> (o) The reference name of the OID for
+the variant set, if one is required. The reference names can be found
+in the <bf/util/ module of <bf/YAZ/.
+
+<tag>class <it/integer class-name/</tag> (m,r) Introduces a new
+class to the variant set.
+
+<tag>type <it/integer type-name datatype/</tag> (m,r) Addes a
+new type to the current class (the one introduced by the most recent
+<bf/class/ directive). The type names belong to the same name space as
+the one used in the tag set definition file.
+</descrip>
+
+The following is an excerpt from the file describing the variant set
+<it/Variant-1/.
+
+<tscreen><verb>
+name variant-1
+reference Variant-1
+
+class 1 variantId
+
+ type 1 variantId octetstring
+
+class 2 body
+
+ type 1 iana string
+ type 2 z39.50 string
+ type 3 other string
+</verb></tscreen>
+
+<sect2>The Element Set (.est) Files
+
+<p>
+The element set specification files describe a selection of a subset
+of the elements of a database record. The element selection mechanism
+is equivalent to the one supplied by the <it/Espec-1/ syntax of the
+Z39.50 specification. In fact, the internal representation of an
+element set specification is identical to the <it/Espec-1/ structure,
+and we'll refer you to the description of that structure for most of
+the detailed semantics of the directives below.
+
+<it>
+NOTE: Not all of the Espec-1 functionality has been implemented yet.
+The fields that are mentioned below all work as expected, unless
+otherwise is noted.
+</it>
+
+The directives available in the element set file are as follows:
+
+<descrip>
+<tag>defaultVariantSetId <it/OID-name/</tag> (o) If variants are used in
+the following, this should provide the name of the variantset used
+(it's not currently possible to specify a different set in the
+individual variant request). In almost all cases (certainly all
+profiles known to us), the name <tt/Variant-1/ should be given here.
+
+<tag>defaultVariantRequest <it/variant-request/</tag> (o) This directive
+provides a default variant request for
+use when the individual element requests (see below) do not contain a
+variant request. Variant requests consist of a blank-separated list of
+variant components. A variant compont is a comma-separated,
+parenthesized triple of variant class, type, and value (the two former
+values being represented as integers). The value can currently only be
+entered as a string (this will change to depend on the definition of
+the variant in question). The special value (@) is interpreted as a
+null value, however.
+
+<tag>simpleElement <it/path ['variant' variant-request]/</tag>
+(o,r) This corresponds to a simple element request in <it/Espec-1/. The
+path consists of a sequence of tag-selectors, where each of these can
+consist of either:
+
+<itemize>
+<item>A simple tag, consisting of a comma-separated type-value pair in
+parenthesis, possibly followed by a colon (:) followed by an
+occurrences-specification (see below). The tag-value can be a number
+or a string. If the first character is an apostrophe ('), this forces
+the value to be interpreted as a string, even if it appears to be numerical.
+
+<item>A WildThing, represented as a question mark (?), possibly
+followed by a colon (:) followed by an occurrences specification (see
+below).
+
+<item>A WildPath, represented as an asterisk (*). Note that the last
+element of the path should not be a wildPath (wildpaths don't work in
+this version).
+</itemize>
+
+The occurrences-specification can be either the string <tt/all/, the
+string <tt/last/, or an explicit value-range. The value-range is
+represented as an integer (the starting point), possibly followed by a
+plus (+) and a second integer (the number of elements, default being
+one).
+
+The variant-request has the same syntax as the defaultVariantRequest
+above. Note that it may sometimes be useful to give an empty variant
+request, simply to disable the default for a specific set of fields
+(we aren't certain if this is proper <it/Espec-1/, but it works in
+this implementation).
+</descrip>
+
+The following is an example of an element specification belonging to
+the GILS profile.
+
+<tscreen><verb>
+simpleelement (1,10)
+simpleelement (1,12)
+simpleelement (2,1)
+simpleelement (1,14)
+simpleelement (4,1)
+simpleelement (4,52)
+</verb></tscreen>
+
+<sect2>The Schema Mapping (.map) Files<label id="schema-mapping">
+
+<p>
+Sometimes, the client might want to receive a database record in
+a schema that differs from the native schema of the record. For
+instance, a client might only know how to process WAIS records, while
+the database record is represented in a more specific schema, such as
+GILS. In this module, a mapping of data to one of the MARC formats is
+also thought of as a schema mapping (mapping the elements of the
+record into fields consistent with the given MARC specification, prior
+to actually converting the data to the ISO2709). This use of the
+object identifier for USMARC as a schema identifier represents an
+overloading of the OID which might not be entirely proper. However,
+it represents the dual role of schema and record syntax which
+is assumed by the MARC family in Z39.50.
+
+<it>
+NOTE: The schema-mapping functions are so far limited to a
+straightforward mapping of elements. This should be extended with
+mechanisms for conversions of the element contents, and conditional
+mappings of elements based on the record contents.
+</it>
+
+These are the directives of the schema mapping file format:
+
+<descrip>
+<tag>targetName <it/name/</tag> (m) A symbolic name for the target schema
+of the table. Useful mostly for diagnostic purposes.
+
+<tag>targetRef <it/OID-name/</tag> (m) An OID name for the target schema.
+This is used, for instance, by a server receiving a request to present
+a record in a different schema from the native one. The name, again,
+is found in the <bf/oid/ module of <bf/YAZ/.
+
+<tag>map <it/element-name target-path/</tag> (o,r) Adds
+an element mapping rule to the table.
+</descrip>
+
+<sect2>The MARC (ISO2709) Representation (.mar) Files
+
+<p>
+This file provides rules for representing a record in the ISO2709
+format. The rules pertain mostly to the values of the constant-length
+header of the record.
+
+<it>NOTE: This will be described better. We're in the process of
+re-evaluating and most likely changing the way that MARC records are
+handled by the system.</it>
+
+<sect1>Exchange Formats
+
+<p>
+Converting records from the internal structure to en exchange format
+is largely an automatic process. Currently, the following exchange
+formats are supported:
+
+<itemize>
+<item>GRS-1. The internal representation is based on GRS-1, so the
+conversion here is straightforward. The system will create
+applied variant and supported variant lists as required, if a record
+contains variant information.
+
+<item>SUTRS. Again, the mapping is fairly straighforward. Indentation
+is used to show the hierarchical structure of the record.
+
+<item>ISO2709-based formats (USMARC, etc.). Only records with a
+two-level structure (corresponding to fields and subfields) can be
+directly mapped to ISO2709. For records with a different structuring
+(eg., GILS), the representation in a structure like USMARC involves a
+schema-mapping (see section <ref id="schema-mapping" name="Schema
+Mapping">), to an &dquot;implied&dquot; USMARC schema (implied,
+because there is no formal schema which specifies the use of the
+USMARC fields outside of ISO2709). The resultant, two-level record is
+then mapped directly from the internal representation to ISO2709. See
+the GILS schema definition files for a detailed example of this
+approach.
+
+<item>Explain. This representation is only available for records
+belonging to the Explain schema.
+
+</itemize>