<chapter id="record-model">
- <!-- $Id: recordmodel.xml,v 1.16 2002-12-30 10:25:24 adam Exp $ -->
- <title>The Record Model</title>
+ <!-- $Id: recordmodel.xml,v 1.23 2006-01-19 09:27:00 marc Exp $ -->
+ <title>GRS Record Model and Filter Modules</title>
- <para>
- The Zebra system is designed to support a wide range of data management
- applications. The system can be configured to handle virtually any
- kind of structured data. Each record in the system is associated with
- a <emphasis>record schema</emphasis> which lends context to the data
- elements of the record.
- Any number of record schemas can coexist in the system.
- Although it may be wise to use only a single schema within
- one database, the system poses no such restrictions.
- </para>
<para>
The record model described in this chapter applies to the fundamental,
structured
record type <literal>grs</literal>, introduced in
- <xref linkend="record-types"/>.
- <!--
- FIXME - Need to describe the simple string-tag model, or at least
- refer to it here. -H
- -->
- </para>
-
- <para>
- Records pass through three different states during processing in the
- system.
+ <xref linkend="componentmodulesgrs"/>.
</para>
- <para>
-
- <itemizedlist>
- <listitem>
-
- <para>
- When records are accessed by the system, they are represented
- in their local, or native format. This might be SGML or HTML files,
- News or Mail archives, MARC records. If the system doesn't already
- know how to read the type of data you need to store, you can set up an
- input filter by preparing conversion rules based on regular
- expressions and possibly augmented by a flexible scripting language
- (Tcl).
- The input filter produces as output an internal representation,
- a tree structure.
-
- </para>
- </listitem>
- <listitem>
-
- <para>
- When records are processed by the system, they are represented
- in a tree-structure, constructed by tagged data elements hanging off a
- root node. The tagged elements may contain data or yet more tagged
- elements in a recursive structure. The system performs various
- actions on this tree structure (indexing, element selection, schema
- mapping, etc.),
-
- </para>
- </listitem>
- <listitem>
-
- <para>
- Before transmitting records to the client, they are first
- converted from the internal structure to a form suitable for exchange
- over the network - according to the Z39.50 standard.
- </para>
- </listitem>
-
- </itemizedlist>
-
- </para>
-
- <sect1 id="local-representation">
- <title>Local Representation</title>
+ <sect1 id="grs-record-filters">
+ <title>GRS Record Filters</title>
<para>
- As mentioned earlier, Zebra places few restrictions on the type of
- data that you can index and manage. Generally, whatever the form of
- the data, it is parsed by an input filter specific to that format, and
- turned into an internal structure that Zebra knows how to handle. This
- process takes place whenever the record is accessed - for indexing and
- retrieval.
- </para>
-
- <para>
- The RecordType parameter in the <literal>zebra.cfg</literal> file, or
- the <literal>-t</literal> option to the indexer tells Zebra how to
- process input records.
- Two basic types of processing are available - raw text and structured
- data. Raw text is just that, and it is selected by providing the
- argument <emphasis>text</emphasis> to Zebra. Structured records are
- all handled internally using the basic mechanisms described in the
- subsequent sections.
- Zebra can read structured records in many different formats.
- How this is done is governed by additional parameters after the
- "grs" keyword, separated by "." characters.
- </para>
-
- <para>
- Four basic subtypes to the <emphasis>grs</emphasis> type are
+ Many basic subtypes of the <emphasis>grs</emphasis> type are
currently available:
</para>
<term>grs.sgml</term>
<listitem>
<para>
- This is the canonical input format —
- described below. It is a simple SGML-like syntax.
+ This is the canonical input format
+ described <xref linkend="grs-canonical-format"/>. It is using
+ simple SGML-like syntax.
+ </para>
+ <!--
+ <para>
+ <literal>libidzebra1.4-mod-grs-sgml not packaged yet ??</literal>
</para>
+ -->
</listitem>
</varlistentry>
<varlistentry>
- <term>grs.regx.<emphasis>filter</emphasis></term>
+ <term>grs.marc<!--.<emphasis>abstract syntax</emphasis>--></term>
<listitem>
<para>
- This enables a user-supplied input
- filter. The mechanisms of these filters are described below.
+ This allows Zebra to read
+ records in the ISO2709 (MARC) encoding standard.
+ <!-- In this case, the
+ last parameter <emphasis>abstract syntax</emphasis> names the
+ <literal>.abs</literal> file (see below)
+ which describes the specific MARC structure of the input record as
+ well as the indexing rules. -->
</para>
+ <para>
+ The loadable <literal>grs.marc</literal> filter module
+ is packaged in the GNU/Debian package
+ <literal>libidzebra1.4-mod-grs-marc</literal>
+ </para>
</listitem>
</varlistentry>
<varlistentry>
- <term>grs.tcl.<emphasis>filter</emphasis></term>
+ <term>grs.marcxml<!--.<emphasis>abstract syntax</emphasis>--></term>
<listitem>
<para>
- Similar to grs.regx but using Tcl for rules.
+ This allows Zebra to read
+ records in the ISO2709??? (MARCXML) encoding standard.
</para>
+ <para>
+ The loadable <literal>grs.marcxml</literal> filter module
+ is also contained in the GNU/Debian package
+ <literal>libidzebra1.4-mod-grs-marc</literal>
+ </para>
</listitem>
</varlistentry>
<varlistentry>
- <term>grs.marc.<emphasis>abstract syntax</emphasis></term>
+ <term>grs.danbib</term>
<listitem>
<para>
- This allows Zebra to read
- records in the ISO2709 (MARC) encoding standard. In this case, the
- last parameter <emphasis>abstract syntax</emphasis> names the
- <literal>.abs</literal> file (see below)
- which describes the specific MARC structure of the input record as
- well as the indexing rules.
+ The <literal>grs.danbib</literal> filter parses DanBib
+ records, a danish MARC record variant called DANMARC.
+ DanBib is the Danish Union Catalogue hosted by the
+ Danish Bibliographic Centre (DBC).
+ </para>
+ <para>The loadable <literal>grs.danbib</literal> filter module
+ is packages in the GNU/Debian package
+ <literal>libidzebra1.4-mod-grs-danbib</literal>.
</para>
</listitem>
</varlistentry>
<term>grs.xml</term>
<listitem>
<para>
- This filter reads XML records. Only one record per file
+ This filter reads XML records and uses <ulink url="http://expat.sourceforge.net/">Expat</ulink> to
+ parse them and convert them into IDZebra's internal
+ <literal>grs</literal> record model.
+ Only one record per file
is supported. The filter is only available if Zebra/YAZ
is compiled with EXPAT support.
</para>
+ <para>
+ The loadable <literal>grs.xml</literal> filter module
+ is packagged in the GNU/Debian package
+ <literal>libidzebra1.4-mod-grs-xml</literal>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>grs.regx<!--.<emphasis>filter</emphasis>--></term>
+ <listitem>
+ <para>
+ This enables a user-supplied Regular Expressions input
+ filter described in
+ <xref linkend="grs-regx-tcl"/>.
+ </para>
+ <para>
+ The loadable <literal>grs.regx</literal> filter module
+ is packaged in the GNU/Debian package
+ <literal>libidzebra1.4-mod-grs-regx</literal>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>grs.tcl<!--.<emphasis>filter</emphasis>--></term>
+ <listitem>
+ <para>
+ Similar to grs.regx but using Tcl for rules, described in
+ <xref linkend="grs-regx-tcl"/>.
+ </para>
+ <para>
+ The loadable <literal>grs.tcl</literal> filter module
+ is also packaged in the GNU/Debian package
+ <literal>libidzebra1.4-mod-grs-regx</literal>
+ </para>
</listitem>
</varlistentry>
</variablelist>
</para>
- <sect2>
- <title>Canonical Input Format</title>
+ <sect2 id="grs-canonical-format">
+ <title>GRS Canonical Input Format</title>
<para>
Although input data can take any form, it is sometimes useful to
makes up the total record. In the canonical input format, the root tag
should contain the name of the schema that lends context to the
elements of the record
- (see <xref linkend="internal-representation"/>).
+ (see <xref linkend="grs-internal-representation"/>).
The following is a GILS record that
contains only a single element (strictly speaking, that makes it an
illegal GILS record, since the GILS profile includes several mandatory
</sect2>
- <sect2>
- <title>Input Filters</title>
+ <sect2 id="grs-regx-tcl">
+ <title>GRS REGX And TCL Input Filters</title>
<para>
In order to handle general input formats, Zebra allows the
<variablelist>
<varlistentry>
- <term>begin <emphasis>type [parameter ... ]</emphasis></term>
+ <term>begin <replaceable>type [parameter ... ]</replaceable></term>
<listitem>
<para>
Begin a new
- data element. The type is one of the following:
+ data element. The <replaceable>type</replaceable> is one of
+ the following:
<variablelist>
<varlistentry>
name of the schema that describes the structure of the record, eg.
<literal>gils</literal> or <literal>wais</literal> (see below).
The <literal>begin record</literal> call should precede
- any other use of the <emphasis>begin</emphasis> statement.
+ any other use of the <replaceable>begin</replaceable> statement.
</para>
</listitem>
</varlistentry>
<listitem>
<para>
Begin a new node in a variant tree. The parameters are
- <emphasis>class type value</emphasis>.
+ <replaceable>class type value</replaceable>.
</para>
</listitem>
</varlistentry>
</listitem>
</varlistentry>
<varlistentry>
- <term>data</term>
+ <term>data <replaceable>parameter</replaceable></term>
<listitem>
<para>
Create a data element. The concatenated arguments make
the layout (whitespace) of the data should be retained for
transmission.
The option <literal>-element</literal>
- <emphasis>tag</emphasis> wraps the data up in
- the <emphasis>tag</emphasis>.
+ <replaceable>tag</replaceable> wraps the data up in
+ the <replaceable>tag</replaceable>.
The use of the <literal>-element</literal> option is equivalent to
- preceding the command with a <emphasis>begin
- element</emphasis> command, and following
- it with the <emphasis>end</emphasis> command.
+ preceding the command with a <replaceable>begin
+ element</replaceable> command, and following
+ it with the <replaceable>end</replaceable> command.
</para>
</listitem>
</varlistentry>
<varlistentry>
- <term>end <emphasis>[type]</emphasis></term>
+ <term>end <replaceable>[type]</replaceable></term>
<listitem>
<para>
Close a tagged element. If no parameter is given,
the last element on the stack is terminated.
The first parameter, if any, is a type name, similar
- to the <emphasis>begin</emphasis> statement.
- For the <emphasis>element</emphasis> type, a tag
+ to the <replaceable>begin</replaceable> statement.
+ For the <replaceable>element</replaceable> type, a tag
name can be provided to terminate a specific tag.
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>unread <replaceable>no</replaceable></term>
+ <listitem>
+ <para>
+ Move the input pointer to the offset of first character that
+ match rule given by <replaceable>no</replaceable>.
+ The first rule from left-to-right is numbered zero,
+ the second rule is named 1 and so on.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</para>
/^Subject:/ BODY /$/ { data -element title $1 }
/^Date:/ BODY /$/ { data -element lastModified $1 }
/\n\n/ BODY END {
- begin element bodyOfDisplay
- begin variant body iana "text/plain"
- data -text $1
- end record
+ begin element bodyOfDisplay
+ begin variant body iana "text/plain"
+ data -text $1
+ end record
}
</screen>
</para>
<para>
- If Zebra is compiled with support for Tcl (Tool Command Language)
- enabled, the statements described above are supplemented with a complete
+ If Zebra is compiled with support for Tcl enabled, the statements
+ described above are supplemented with a complete
scripting environment, including control structures (conditional
expressions and loop constructs), and powerful string manipulation
- mechanisms for modifying the elements of a record. Tcl is a popular
- scripting environment, with several tutorials available both online
- and in hardcopy.
+ mechanisms for modifying the elements of a record.
</para>
</sect2>
</sect1>
- <sect1 id="internal-representation">
- <title>Internal Representation</title>
+ <sect1 id="grs-internal-representation">
+ <title>GRS Internal Record Representation</title>
<para>
When records are manipulated by the system, they're represented in a
</sect1>
- <sect1 id="data-model">
- <title>Configuring Your Data Model</title>
+ <sect1 id="grs-record-model">
+ <title>GRS Record Model Configuration</title>
<para>
The following sections describe the configuration files that govern
- the internal management of data records. The system searches for the files
+ the internal management of <literal>grs</literal> records.
+ The system searches for the files
in the directories specified by the <emphasis>profilePath</emphasis>
setting in the <literal>zebra.cfg</literal> file.
</para>
</para>
</listitem>
</varlistentry>
-
+ <varlistentry>
+ <term>melm <replaceable>field$subfield attributes</replaceable></term>
+ <listitem>
+ <para>
+ This directive is specifically for MARC-formatted records,
+ ingested either in the form of MARCXML documents, or in the
+ ISO2709/Z39.2 format using the grs.marcxml input filter. You can
+ specify indexing rules for any subfield, or you can leave off the
+ <replaceable>$subfield</replaceable> part and specify default rules
+ for all subfields of the given field (note: default rules should come
+ after any subfield-specific rules in the configuration file). The
+ <replaceable>attributes</replaceable> have the same syntax and meaning
+ as for the 'elm' directive above.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry>
<term>encoding <replaceable>encodingname</replaceable></term>
<listitem>
esetname G gils-g.est
esetname F @
- elm (1,10) rank -
- elm (1,12) url -
- elm (1,14) localControlNumber Local-number
- elm (1,16) dateOfLastModification Date/time-last-modified
- elm (2,1) title w:!,p:!
- elm (4,1) controlIdentifier Identifier-standard
- elm (2,6) abstract Abstract
- elm (4,51) purpose !
- elm (4,52) originator -
- elm (4,53) accessConstraints !
- elm (4,54) useConstraints !
- elm (4,70) availability -
- elm (4,70)/(4,90) distributor -
- elm (4,70)/(4,90)/(2,7) distributorName !
- elm (4,70)/(4,90)/(2,10 distributorOrganization !
- elm (4,70)/(4,90)/(4,2) distributorStreetAddress !
- elm (4,70)/(4,90)/(4,3) distributorCity !
+ elm (1,10) rank -
+ elm (1,12) url -
+ elm (1,14) localControlNumber Local-number
+ elm (1,16) dateOfLastModification Date/time-last-modified
+ elm (2,1) title w:!,p:!
+ elm (4,1) controlIdentifier Identifier-standard
+ elm (2,6) abstract Abstract
+ elm (4,51) purpose !
+ elm (4,52) originator -
+ elm (4,53) accessConstraints !
+ elm (4,54) useConstraints !
+ elm (4,70) availability -
+ elm (4,70)/(4,90) distributor -
+ elm (4,70)/(4,90)/(2,7) distributorName !
+ elm (4,70)/(4,90)/(2,10) distributorOrganization !
+ elm (4,70)/(4,90)/(4,2) distributorStreetAddress !
+ elm (4,70)/(4,90)/(4,3) distributorCity !
</screen>
</para>
special-purpose fields such as WWW-style linkages (URx).
</para>
- <para>
- The field types, and hence character sets, are associated with data
- elements by the .abs files (see above).
- The file <literal>default.idx</literal>
- provides the association between field type codes (as used in the .abs
- files) and the character map files (with the .chr suffix). The format
- of the .idx file is as follows
- </para>
-
- <para>
- <variablelist>
-
- <varlistentry>
- <term>index <emphasis>field type code</emphasis></term>
- <listitem>
- <para>
- This directive introduces a new search index code.
- The argument is a one-character code to be used in the
- .abs files to select this particular index type. An index, roughly,
- corresponds to a particular structure attribute during search. Refer
- to <xref linkend="search"/>.
- </para>
- </listitem></varlistentry>
- <varlistentry>
- <term>sort <emphasis>field code type</emphasis></term>
- <listitem>
- <para>
- This directive introduces a
- sort index. The argument is a one-character code to be used in the
- .abs fie to select this particular index type. The corresponding
- use attribute must be used in the sort request to refer to this
- particular sort index. The corresponding character map (see below)
- is used in the sort process.
- </para>
- </listitem></varlistentry>
- <varlistentry>
- <term>completeness <emphasis>boolean</emphasis></term>
- <listitem>
- <para>
- This directive enables or disables complete field indexing.
- The value of the <emphasis>boolean</emphasis> should be 0
- (disable) or 1. If completeness is enabled, the index entry will
- contain the complete contents of the field (up to a limit), with words
- (non-space characters) separated by single space characters
- (normalized to " " on display). When completeness is
- disabled, each word is indexed as a separate entry. Complete subfield
- indexing is most useful for fields which are typically browsed (eg.
- titles, authors, or subjects), or instances where a match on a
- complete subfield is essential (eg. exact title searching). For fields
- where completeness is disabled, the search engine will interpret a
- search containing space characters as a word proximity search.
- </para>
- </listitem></varlistentry>
- <varlistentry>
- <term>charmap <emphasis>filename</emphasis></term>
- <listitem>
- <para>
- This is the filename of the character
- map to be used for this index for field type.
- </para>
- </listitem></varlistentry>
- </variablelist>
- </para>
-
- <para>
- The contents of the character map files are structured as follows:
- </para>
-
- <para>
- <variablelist>
-
- <varlistentry>
- <term>lowercase <emphasis>value-set</emphasis></term>
- <listitem>
- <para>
- This directive introduces the basic value set of the field type.
- The format is an ordered list (without spaces) of the
- characters which may occur in "words" of the given type.
- The order of the entries in the list determines the
- sort order of the index. In addition to single characters, the
- following combinations are legal:
- </para>
-
- <para>
-
- <itemizedlist>
- <listitem>
- <para>
- Backslashes may be used to introduce three-digit octal, or
- two-digit hex representations of single characters
- (preceded by <literal>x</literal>).
- In addition, the combinations
- \\, \\r, \\n, \\t, \\s (space — remember that real
- space-characters may not occur in the value definition), and
- \\ are recognized, with their usual interpretation.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Curly braces {} may be used to enclose ranges of single
- characters (possibly using the escape convention described in the
- preceding point), eg. {a-z} to introduce the
- standard range of ASCII characters.
- Note that the interpretation of such a range depends on
- the concrete representation in your local, physical character set.
- </para>
- </listitem>
-
- <listitem>
- <para>
- paranthesises () may be used to enclose multi-byte characters -
- eg. diacritics or special national combinations (eg. Spanish
- "ll"). When found in the input stream (or a search term),
- these characters are viewed and sorted as a single character, with a
- sorting value depending on the position of the group in the value
- statement.
- </para>
- </listitem>
+ <sect3 id="default-idx-file">
+ <title>The default.idx file</title>
+ <para>
+ The field types, and hence character sets, are associated with data
+ elements by the .abs files (see above).
+ The file <literal>default.idx</literal>
+ provides the association between field type codes (as used in the .abs
+ files) and the character map files (with the .chr suffix). The format
+ of the .idx file is as follows
+ </para>
- </itemizedlist>
+ <para>
+ <variablelist>
+
+ <varlistentry>
+ <term>index <emphasis>field type code</emphasis></term>
+ <listitem>
+ <para>
+ This directive introduces a new search index code.
+ The argument is a one-character code to be used in the
+ .abs files to select this particular index type. An index, roughly,
+ corresponds to a particular structure attribute during search. Refer
+ to <xref linkend="search"/>.
+ </para>
+ </listitem></varlistentry>
+ <varlistentry>
+ <term>sort <emphasis>field code type</emphasis></term>
+ <listitem>
+ <para>
+ This directive introduces a
+ sort index. The argument is a one-character code to be used in the
+ .abs fie to select this particular index type. The corresponding
+ use attribute must be used in the sort request to refer to this
+ particular sort index. The corresponding character map (see below)
+ is used in the sort process.
+ </para>
+ </listitem></varlistentry>
+ <varlistentry>
+ <term>completeness <emphasis>boolean</emphasis></term>
+ <listitem>
+ <para>
+ This directive enables or disables complete field indexing.
+ The value of the <emphasis>boolean</emphasis> should be 0
+ (disable) or 1. If completeness is enabled, the index entry will
+ contain the complete contents of the field (up to a limit), with words
+ (non-space characters) separated by single space characters
+ (normalized to " " on display). When completeness is
+ disabled, each word is indexed as a separate entry. Complete subfield
+ indexing is most useful for fields which are typically browsed (eg.
+ titles, authors, or subjects), or instances where a match on a
+ complete subfield is essential (eg. exact title searching). For fields
+ where completeness is disabled, the search engine will interpret a
+ search containing space characters as a word proximity search.
+ </para>
+ </listitem></varlistentry>
+ <varlistentry>
+ <term>charmap <emphasis>filename</emphasis></term>
+ <listitem>
+ <para>
+ This is the filename of the character
+ map to be used for this index for field type.
+ </para>
+ </listitem></varlistentry>
+ </variablelist>
+ </para>
+ </sect3>
- </para>
- </listitem></varlistentry>
- <varlistentry>
- <term>uppercase <emphasis>value-set</emphasis></term>
- <listitem>
- <para>
- This directive introduces the
- upper-case equivalencis to the value set (if any). The number and
- order of the entries in the list should be the same as in the
- <literal>lowercase</literal> directive.
- </para>
- </listitem></varlistentry>
- <varlistentry>
- <term>space <emphasis>value-set</emphasis></term>
- <listitem>
- <para>
- This directive introduces the character
- which separate words in the input stream. Depending on the
- completeness mode of the field in question, these characters either
- terminate an index entry, or delimit individual "words" in
- the input stream. The order of the elements is not significant —
- otherwise the representation is the same as for the
- <literal>uppercase</literal> and <literal>lowercase</literal>
- directives.
- </para>
- </listitem></varlistentry>
- <varlistentry>
- <term>map <emphasis>value-set</emphasis>
- <emphasis>target</emphasis></term>
- <listitem>
- <para>
- This directive introduces a
- mapping between each of the members of the value-set on the left to
- the character on the right. The character on the right must occur in
- the value set (the <literal>lowercase</literal> directive) of
- the character set, but
- it may be a paranthesis-enclosed multi-octet character. This directive
- may be used to map diacritics to their base characters, or to map
- HTML-style character-representations to their natural form, etc.
- </para>
- </listitem></varlistentry>
- </variablelist>
- </para>
+ <sect3 id="character-map-files">
+ <title>The character map file format</title>
+ <para>
+ The contents of the character map files are structured as follows:
+ </para>
+ <para>
+ <variablelist>
+
+ <varlistentry>
+ <term>lowercase <emphasis>value-set</emphasis></term>
+ <listitem>
+ <para>
+ This directive introduces the basic value set of the field type.
+ The format is an ordered list (without spaces) of the
+ characters which may occur in "words" of the given type.
+ The order of the entries in the list determines the
+ sort order of the index. In addition to single characters, the
+ following combinations are legal:
+ </para>
+
+ <para>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ Backslashes may be used to introduce three-digit octal, or
+ two-digit hex representations of single characters
+ (preceded by <literal>x</literal>).
+ In addition, the combinations
+ \\, \\r, \\n, \\t, \\s (space — remember that real
+ space-characters may not occur in the value definition), and
+ \\ are recognized, with their usual interpretation.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Curly braces {} may be used to enclose ranges of single
+ characters (possibly using the escape convention described in the
+ preceding point), eg. {a-z} to introduce the
+ standard range of ASCII characters.
+ Note that the interpretation of such a range depends on
+ the concrete representation in your local, physical character set.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ paranthesises () may be used to enclose multi-byte characters -
+ eg. diacritics or special national combinations (eg. Spanish
+ "ll"). When found in the input stream (or a search term),
+ these characters are viewed and sorted as a single character, with a
+ sorting value depending on the position of the group in the value
+ statement.
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+ </listitem></varlistentry>
+ <varlistentry>
+ <term>uppercase <emphasis>value-set</emphasis></term>
+ <listitem>
+ <para>
+ This directive introduces the
+ upper-case equivalencis to the value set (if any). The number and
+ order of the entries in the list should be the same as in the
+ <literal>lowercase</literal> directive.
+ </para>
+ </listitem></varlistentry>
+ <varlistentry>
+ <term>space <emphasis>value-set</emphasis></term>
+ <listitem>
+ <para>
+ This directive introduces the character
+ which separate words in the input stream. Depending on the
+ completeness mode of the field in question, these characters either
+ terminate an index entry, or delimit individual "words" in
+ the input stream. The order of the elements is not significant —
+ otherwise the representation is the same as for the
+ <literal>uppercase</literal> and <literal>lowercase</literal>
+ directives.
+ </para>
+ </listitem></varlistentry>
+ <varlistentry>
+ <term>map <emphasis>value-set</emphasis>
+ <emphasis>target</emphasis></term>
+ <listitem>
+ <para>
+ This directive introduces a mapping between each of the
+ members of the value-set on the left to the character on the
+ right. The character on the right must occur in the value
+ set (the <literal>lowercase</literal> directive) of the
+ character set, but it may be a paranthesis-enclosed
+ multi-octet character. This directive may be used to map
+ diacritics to their base characters, or to map HTML-style
+ character-representations to their natural form, etc. The
+ map directive can also be used to ignore leading articles in
+ searching and/or sorting, and to perform other special
+ transformations. See section <xref
+ linkend="leading-articles"/>.
+ </para>
+ </listitem></varlistentry>
+ </variablelist>
+ </para>
+ </sect3>
+ <sect3 id="leading-articles">
+ <title>Ignoring leading articles</title>
+ <para>
+ In addition to specifying sort orders, space (blank) handling,
+ and upper/lowercase folding, you can also use the character map
+ files to make Zebra ignore leading articles in sorting records,
+ or when doing complete field searching.
+ </para>
+ <para>
+ This is done using the <literal>map</literal> directive in the
+ character map file. In a nutshell, what you do is map certain
+ sequences of characters, when they occur <emphasis> in the
+ beginning of a field</emphasis>, to a space. Assuming that the
+ character "@" is defined as a space character in your file, you
+ can do:
+ <screen>
+ map (^The\s) @
+ map (^the\s) @
+ </screen>
+ The effect of these directives is to map either 'the' or 'The',
+ followed by a space character, to a space. The hat ^ character
+ denotes beginning-of-field only when complete-subfield indexing
+ or sort indexing is taking place; otherwise, it is treated just
+ as any other character.
+ </para>
+ <para>
+ Because the <literal>default.idx</literal> file can be used to
+ associate different character maps with different indexing types
+ -- and you can create additional indexing types, should the need
+ arise -- it is possible to specify that leading articles should
+ be ignored either in sorting, in complete-field searching, or
+ both.
+ </para>
+ <para>
+ If you ignore certain prefixes in sorting, then these will be
+ eliminated from the index, and sorting will take place as if
+ they weren't there. However, if you set the system up to ignore
+ certain prefixes in <emphasis>searching</emphasis>, then these
+ are deleted both from the indexes and from query terms, when the
+ client specifies complete-field searching. This has the effect
+ that a search for 'the science journal' and 'science journal'
+ would both produce the same results.
+ </para>
+ </sect3>
</sect2>
-
</sect1>
- <sect1 id="formats">
- <title>Exchange Formats</title>
+ <sect1 id="grs-exchange-formats">
+ <title>GRS Exchange Formats</title>
<para>
- Converting records from the internal structure to en exchange format
+ Converting records from the internal structure to an exchange format
is largely an automatic process. Currently, the following exchange
formats are supported:
</para>
</para>
</listitem>
+ <!-- FIXME - Is this used anywhere ? -H -->
<listitem>
<para>
SOIF. Support for this syntax is experimental, and is currently
abstract syntaxes can be mapped to the SOIF format, although nested
elements are represented by concatenation of the tag names at each
level.
- <!-- FIXME - Is this used anywhere ? -H -->
</para>
</listitem>
-
+
</itemizedlist>
</para>
</sect1>
projects / idzebra-moved-to-github.git / blobdiff