<chapter id="administration">
- <!-- $Id: administration.xml,v 1.44 2006-09-05 12:01:31 adam Exp $ -->
- <title>Administrating Zebra</title>
+ <title>Administrating &zebra;</title>
<!-- ### It's a bit daft that this chapter (which describes half of
the configuration-file formats) is separated from
"recordmodel-grs.xml" (which describes the other half) by the
-->
<para>
- Unlike many simpler retrieval systems, Zebra supports safe, incremental
+ Unlike many simpler retrieval systems, &zebra; supports safe, incremental
updates to an existing index.
</para>
<para>
- Normally, when Zebra modifies the index it reads a number of records
+ Normally, when &zebra; modifies the index it reads a number of records
that you specify.
Depending on your specifications and on the contents of each record
one the following events take place for each record:
<listitem>
<para>
The record is indexed as if it never occurred before.
- Either the Zebra system doesn't know how to identify the record or
- Zebra can identify the record but didn't find it to be already indexed.
+ Either the &zebra; system doesn't know how to identify the record or
+ &zebra; can identify the record but didn't find it to be already indexed.
</para>
</listitem>
</varlistentry>
</para>
<para>
- Please note that in both the modify- and delete- case the Zebra
+ Please note that in both the modify- and delete- case the &zebra;
indexer must be able to generate a unique key that identifies the record
in question (more on this below).
</para>
<para>
- To administrate the Zebra retrieval system, you run the
+ To administrate the &zebra; retrieval system, you run the
<literal>zebraidx</literal> program.
This program supports a number of options which are preceded by a dash,
and a few commands (not preceded by dash).
</para>
<para>
- Both the Zebra administrative tool and the Z39.50 server share a
+ Both the &zebra; administrative tool and the &acro.z3950; server share a
set of index files and a global configuration file.
The name of the configuration file defaults to
<literal>zebra.cfg</literal>.
Indexing is a per-record process, in which either insert/modify/delete
will occur. Before a record is indexed search keys are extracted from
whatever might be the layout the original record (sgml,html,text, etc..).
- The Zebra system currently supports two fundamental types of records:
+ The &zebra; system currently supports two fundamental types of records:
structured and simple text.
To specify a particular extraction process, use either the
command line option <literal>-t</literal> or specify a
</sect1>
<sect1 id="zebra-cfg">
- <title>The Zebra Configuration File</title>
+ <title>The &zebra; Configuration File</title>
<para>
- The Zebra configuration file, read by <literal>zebraidx</literal> and
+ The &zebra; configuration file, read by <literal>zebraidx</literal> and
<literal>zebrasrv</literal> defaults to <literal>zebra.cfg</literal>
unless specified by <literal>-c</literal> option.
</para>
In the configuration file, the group name is placed before the option
name itself, separated by a dot (.). For instance, to set the record type
for group <literal>public</literal> to <literal>grs.sgml</literal>
- (the SGML-like format for structured records) you would write:
+ (the &acro.sgml;-like format for structured records) you would write:
</para>
<para>
<replaceable>database</replaceable></term>
<listitem>
<para>
- Specifies the Z39.50 database name.
+ Specifies the &acro.z3950; database name.
<!-- FIXME - now we can have multiple databases in one server. -H -->
</para>
</listitem>
<listitem>
<para>
Specifies whether the records should be stored internally
- in the Zebra system files.
+ in the &zebra; system files.
If you want to maintain the raw records yourself,
this option should be false (0).
- If you want Zebra to take care of the records for you, it
+ If you want &zebra; to take care of the records for you, it
should be true(1).
</para>
</listitem>
<term>register: <replaceable>register-location</replaceable></term>
<listitem>
<para>
- Specifies the location of the various register files that Zebra uses
+ Specifies the location of the various register files that &zebra; uses
to represent your databases.
See <xref linkend="register-location"/>.
</para>
<term>shadow: <replaceable>register-location</replaceable></term>
<listitem>
<para>
- Enables the <emphasis>safe update</emphasis> facility of Zebra, and
+ Enables the <emphasis>safe update</emphasis> facility of &zebra;, and
tells the system where to place the required, temporary files.
See <xref linkend="shadow-registers"/>.
</para>
<para>
Specifies a path of profile specification files.
The path is composed of one or more directories separated by
- colon. Similar to PATH for UNIX systems.
+ colon. Similar to <literal>PATH</literal> for UNIX systems.
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>modulePath: <replaceable>path</replaceable></term>
+ <listitem>
+ <para>
+ Specifies a path of record filter modules.
+ The path is composed of one or more directories separated by
+ colon. Similar to <literal>PATH</literal> for UNIX systems.
+ The 'make install' procedure typically puts modules in
+ <filename>/usr/local/lib/idzebra-2.0/modules</filename>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>index: <replaceable>filename</replaceable></term>
+ <listitem>
+ <para>
+ Defines the filename which holds fields structure
+ definitions. If omitted, the file <filename>default.idx</filename>
+ is read.
+ Refer to <xref linkend="default-idx-file"/> for
+ more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sortmax: <replaceable>integer</replaceable></term>
+ <listitem>
+ <para>
+ Specifies the maximum number of records that will be sorted
+ in a result set. If the result set contains more than
+ <replaceable>integer</replaceable> records, records after the
+ limit will not be sorted. If omitted, the default value is
+ 1,000.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>staticrank: <replaceable>integer</replaceable></term>
+ <listitem>
+ <para>
+ Enables whether static ranking is to be enabled (1) or
+ disabled (0). If omitted, it is disabled - corresponding
+ to a value of 0.
+ Refer to <xref linkend="administration-ranking-static"/> .
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term>estimatehits: <replaceable>integer</replaceable></term>
+ <listitem>
+ <para>
+ Controls whether &zebra; should calculate approximate hit counts and
+ at which hit count it is to be enabled.
+ A value of 0 disables approximate hit counts.
+ For a positive value approximate hit count is enabled
+ if it is known to be larger than <replaceable>integer</replaceable>.
+ </para>
+ <para>
+ Approximate hit counts can also be triggered by a particular
+ attribute in a query.
+ Refer to <xref linkend="querymodel-zebra-global-attr-limit"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term>attset: <replaceable>filename</replaceable></term>
<listitem>
<para>
- Specifies the filename(s) of attribute set files for use in
- searching. At least the Bib-1 set should be loaded
- (<literal>bib1.att</literal>).
- The <literal>profilePath</literal> setting is used to look for
- the specified files.
- See <xref linkend="attset-files"/>
+ Specifies the filename(s) of attribute set files for use in
+ searching. In many configurations <filename>bib1.att</filename>
+ is used, but that is not required. If Classic Explain
+ attributes is to be used for searching,
+ <filename>explain.att</filename> must be given.
+ The path to att-files in general can be given using
+ <literal>profilePath</literal> setting.
+ See also <xref linkend="attset-files"/>.
</para>
</listitem>
</varlistentry>
<term>root: <replaceable>dir</replaceable></term>
<listitem>
<para>
- Specifies a directory base for Zebra. All relative paths
+ Specifies a directory base for &zebra;. All relative paths
given (in profilePath, register, shadow) are based on this
- directory. This setting is useful if your Zebra server
+ directory. This setting is useful if your &zebra; server
is running in a different directory from where
<literal>zebra.cfg</literal> is located.
</para>
<term>passwd: <replaceable>file</replaceable></term>
<listitem>
<para>
- Specifies a file with description of user accounts for Zebra.
+ Specifies a file with description of user accounts for &zebra;.
The format is similar to that known to Apache's htpasswd files
and UNIX' passwd files. Non-empty lines not beginning with
# are considered account lines. There is one account per-line.
<term>passwd.c: <replaceable>file</replaceable></term>
<listitem>
<para>
- Specifies a file with description of user accounts for Zebra.
+ Specifies a file with description of user accounts for &zebra;.
File format is similar to that used by the passwd directive except
that the password are encrypted. Use Apache's htpasswd or similar
for maintenance.
<replaceable>permstring</replaceable></term>
<listitem>
<para>
- Specifies permissions (priviledge) for a user that are allowed
- to access Zebra via the passwd system. There are two kinds
+ Specifies permissions (privilege) for a user that are allowed
+ to access &zebra; via the passwd system. There are two kinds
of permissions currently: read (r) and write(w). By default
users not listed in a permission directive are given the read
privilege. To specify permissions for a user with no
- username, or Z39.50 anonymous style use
+ username, or &acro.z3950; anonymous style use
<literal>anonymous</literal>. The permstring consists of
a sequence of characters. Include character <literal>w</literal>
- for write/update access, <literal>r</literal> for read access.
+ for write/update access, <literal>r</literal> for read access and
+ <literal>a</literal> to allow anonymous access through this account.
</para>
</listitem>
</varlistentry>
<varlistentry>
- <term>dbaccess <replaceable>accessfile</replaceable></term>
+ <term>dbaccess: <replaceable>accessfile</replaceable></term>
<listitem>
<para>
Names a file which lists database subscriptions for individual users.
- The access file should consists of lines of the form <literal>username:
- dbnames</literal>, where dbnames is a list of database names, seprated by
- '+'. No whitespace is allowed in the database list.
+ The access file should consists of lines of the form
+ <literal>username: dbnames</literal>, where dbnames is a list of
+ database names, separated by '+'. No whitespace is allowed in the
+ database list.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>encoding: <replaceable>charsetname</replaceable></term>
+ <listitem>
+ <para>
+ Tells &zebra; to interpret the terms in Z39.50 queries as
+ having been encoded using the specified character
+ encoding. The default is <literal>ISO-8859-1</literal>; one
+ useful alternative is <literal>UTF-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>storeKeys: <replaceable>value</replaceable></term>
+ <listitem>
+ <para>
+ Specifies whether &zebra; keeps a copy of indexed keys.
+ Use a value of 1 to enable; 0 to disable. If storeKeys setting is
+ omitted, it is enabled. Enabled storeKeys
+ are required for updating and deleting records. Disable only
+ storeKeys to save space and only plan to index data once.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>storeData: <replaceable>value</replaceable></term>
+ <listitem>
+ <para>
+ Specifies whether &zebra; keeps a copy of indexed records.
+ Use a value of 1 to enable; 0 to disable. If storeData setting is
+ omitted, it is enabled. A storeData setting of 0 (disabled) makes
+ Zebra fetch records from the original locaction in the file
+ system using filename, file offset and file length. For the
+ DOM and ALVIS filter, the storeData setting is ignored.
</para>
</listitem>
</varlistentry>
<title>Locating Records</title>
<para>
- The default behavior of the Zebra system is to reference the
+ The default behavior of the &zebra; system is to reference the
records from their original location, i.e. where they were found when you
run <literal>zebraidx</literal>.
That is, when a client wishes to retrieve a record
If your input files are not permanent - for example if you retrieve
your records from an outside source, or if they were temporarily
mounted on a CD-ROM drive,
- you may want Zebra to make an internal copy of them. To do this,
+ you may want &zebra; to make an internal copy of them. To do this,
you specify 1 (true) in the <literal>storeData</literal> setting. When
- the Z39.50 server retrieves the records they will be read from the
+ the &acro.z3950; server retrieves the records they will be read from the
internal file structures of the system.
</para>
<para>
Consider a system in which you have a group of text files called
<literal>simple</literal>.
- That group of records should belong to a Z39.50 database called
+ That group of records should belong to a &acro.z3950; database called
<literal>textbase</literal>.
The following <literal>zebra.cfg</literal> file will suffice:
</para>
To enable indexing with pathname IDs, you must specify
<literal>file</literal> as the value of <literal>recordId</literal>
in the configuration file. In addition, you should set
- <literal>storeKeys</literal> to <literal>1</literal>, since the Zebra
+ <literal>storeKeys</literal> to <literal>1</literal>, since the &zebra;
indexer must save additional information about the contents of each record
in order to modify the indexes correctly at a later time.
</para>
<note>
<para>You cannot start out with a group of records with simple
indexing (no record IDs as in the previous section) and then later
- enable file record Ids. Zebra must know from the first time that you
+ enable file record Ids. &zebra; must know from the first time that you
index the group that
the files should be indexed with file record IDs.
</para>
information. If you have a group of records that explicitly associates
an ID with each record, this method is convenient. For example, the
record format may contain a title or a ID-number - unique within the group.
- In either case you specify the Z39.50 attribute set and use-attribute
+ In either case you specify the &acro.z3950; attribute set and use-attribute
location in which this information is stored, and the system looks at
that field to determine the identity of the record.
</para>
</para>
<para>
- For instance, the sample GILS records that come with the Zebra
+ For instance, the sample GILS records that come with the &zebra;
distribution contain a unique ID in the data tagged Control-Identifier.
- The data is mapped to the Bib-1 use attribute Identifier-standard
+ The data is mapped to the &acro.bib1; use attribute Identifier-standard
(code 1007). To use this field as a record id, specify
<literal>(bib1,Identifier-standard)</literal> as the value of the
<literal>recordId</literal> in the configuration file.
<literal>zebraidx</literal>. If you wish to store these, possibly large,
files somewhere else, you must add the <literal>register</literal>
entry to the <literal>zebra.cfg</literal> file.
- Furthermore, the Zebra system allows its file
+ Furthermore, the &zebra; system allows its file
structures to span multiple file systems, which is useful for
managing very large databases.
</para>
The value of the <literal>register</literal> setting is a sequence
of tokens. Each token takes the form:
- <screen>
- <emphasis>dir</emphasis><literal>:</literal><emphasis>size</emphasis>.
- </screen>
+ <emphasis>dir</emphasis><literal>:</literal><emphasis>size</emphasis>
The <emphasis>dir</emphasis> specifies a directory in which index files
will be stored and the <emphasis>size</emphasis> specifies the maximum
- size of all files in that directory. The Zebra indexer system fills
+ size of all files in that directory. The &zebra; indexer system fills
each directory in the order specified and use the next specified
directories as needed.
The <emphasis>size</emphasis> is an integer followed by a qualifier
<literal>k</literal> for kilobytes.
<literal>M</literal> for megabytes,
<literal>G</literal> for gigabytes.
+ Specifying a negative value disables the checking (it still needs the unit,
+ use <literal>-1b</literal>).
</para>
<para>
- For instance, if you have allocated two disks for your register, and
+ For instance, if you have allocated three disks for your register, and
the first disk is mounted
- on <literal>/d1</literal> and has 2GB of free space and the
- second, mounted on <literal>/d2</literal> has 3.6 GB, you could
- put this entry in your configuration file:
+ on <literal>/d1</literal> and has 2GB of free space, the
+ second, mounted on <literal>/d2</literal> has 3.6 GB, and the third,
+ on which you have more space than you bother to worry about, mounted on
+ <literal>/d3</literal> you could put this entry in your configuration file:
<screen>
- register: /d1:2G /d2:3600M
+ register: /d1:2G /d2:3600M /d3:-1b
</screen>
-
</para>
<para>
- Note that Zebra does not verify that the amount of space specified is
+ Note that &zebra; does not verify that the amount of space specified is
actually available on the directory (file system) specified - it is
your responsibility to ensure that enough space is available, and that
other applications do not attempt to use the free space. In a large
production system, it is recommended that you allocate one or more
- file system exclusively to the Zebra register files.
+ file system exclusively to the &zebra; register files.
</para>
</sect1>
<title>Description</title>
<para>
- The Zebra server supports <emphasis>updating</emphasis> of the index
+ The &zebra; server supports <emphasis>updating</emphasis> of the index
structures. That is, you can add, modify, or remove records from
- databases managed by Zebra without rebuilding the entire index.
+ databases managed by &zebra; without rebuilding the entire index.
Since this process involves modifying structured files with various
references between blocks of data in the files, the update process
is inherently sensitive to system crashes, or to process interruptions:
<para>
You can solve these problems by enabling the shadow register system in
- Zebra.
+ &zebra;.
During the updating procedure, <literal>zebraidx</literal> will temporarily
write changes to the involved files in a set of "shadow
files", without modifying the files that are accessed by the
<title>Overview</title>
<para>
The default ordering of a result set is left up to the server,
- which inside Zebra means sorting in ascending document ID order.
+ which inside &zebra; means sorting in ascending document ID order.
This is not always the order humans want to browse the sometimes
quite large hit sets. Ranking and sorting comes to the rescue.
</para>
Simply put, <literal>dynamic relevance ranking</literal>
sorts a set of retrieved records such that those most likely to be
relevant to your request are retrieved first.
- Internally, Zebra retrieves all documents that satisfy your
+ Internally, &zebra; retrieves all documents that satisfy your
query, and re-orders the hit list to arrange them based on
a measurement of similarity between your query and the content of
each record.
<title>Static Ranking</title>
<para>
- Zebra uses internally inverted indexes to look up term occurencies
+ &zebra; uses internally inverted indexes to look up term frequencies
in documents. Multiple queries from different indexes can be
combined by the binary boolean operations <literal>AND</literal>,
<literal>OR</literal> and/or <literal>NOT</literal> (which
<screen>
staticrank: 1
</screen>
- directive in the main core Zebra configuration file, the internal document
+ directive in the main core &zebra; configuration file, the internal document
keys used for ordering are augmented by a preceding integer, which
contains the static rank of a given document, and the index lists
are ordered
</para>
<para>
The experimental <literal>alvis</literal> filter provides a
- directive to fetch static rank information out of the indexed XML
+ directive to fetch static rank information out of the indexed &acro.xml;
records, thus making <emphasis>all</emphasis> hit sets ordered
after <emphasis>ascending</emphasis> static
rank, and for those doc's which have the same static rank, ordered
indexing time (this is why we
call it ``dynamic ranking'' in the first place ...)
It is invoked by adding
- the Bib-1 relation attribute with
- value ``relevance'' to the PQF query (that is,
+ the &acro.bib1; relation attribute with
+ value ``relevance'' to the &acro.pqf; query (that is,
<literal>@attr 2=102</literal>, see also
<ulink url="&url.z39.50;bib1.html">
- The BIB-1 Attribute Set Semantics</ulink>, also in
+ The &acro.bib1; Attribute Set Semantics</ulink>, also in
<ulink url="&url.z39.50.attset.bib1;">HTML</ulink>).
To find all articles with the word <literal>Eoraptor</literal> in
- the title, and present them relevance ranked, issue the PQF query:
+ the title, and present them relevance ranked, issue the &acro.pqf; query:
<screen>
@attr 2=102 @attr 1=4 Eoraptor
</screen>
</para>
<sect3 id="administration-ranking-dynamic-rank1">
- <title>Dynamically ranking using PQF queries with the 'rank-1'
+ <title>Dynamically ranking using &acro.pqf; queries with the 'rank-1'
algorithm</title>
<para>
The default <literal>rank-1</literal> ranking module implements a
TF/IDF (Term Frequecy over Inverse Document Frequency) like
- algorithm. In contrast to the usual defintion of TF/IDF
+ algorithm. In contrast to the usual definition of TF/IDF
algorithms, which only considers searching in one full-text
index, this one works on multiple indexes at the same time.
More precisely,
- Zebra does boolean queries and searches in specific addressed
+ &zebra; does boolean queries and searches in specific addressed
indexes (there are inverted indexes pointing from terms in the
dictionary to documents and term positions inside documents).
It works like this:
<term>Query Components</term>
<listitem>
<para>
- First, the boolean query is dismantled into it's principal components,
+ First, the boolean query is dismantled into its principal components,
i.e. atomic queries where one term is looked up in one index.
For example, the query
<screen>
</para>
<para>
It is possible to apply dynamic ranking on only parts of the
- PQF query:
+ &acro.pqf; query:
<screen>
@and @attr 2=102 @attr 1=1010 Utah @attr 1=1018 Springer
</screen>
</para>
<para>
Ranking weights may be used to pass a value to a ranking
- algorithm, using the non-standard BIB-1 attribute type 9.
+ algorithm, using the non-standard &acro.bib1; attribute type 9.
This allows one branch of a query to use one value while
another branch uses a different one. For example, we can search
for <literal>utah</literal> in the
</para>
<para>
The default weight is
- sqrt(1000) ~ 34 , as the Z39.50 standard prescribes that the top score
+ sqrt(1000) ~ 34 , as the &acro.z3950; standard prescribes that the top score
is 1000 and the bottom score is 0, encoded in integers.
</para>
<warning>
<!--
<sect3 id="administration-ranking-dynamic-rank1">
- <title>Dynamically ranking PQF queries with the 'rank-static'
+ <title>Dynamically ranking &acro.pqf; queries with the 'rank-static'
algorithm</title>
<para>
The dummy <literal>rank-static</literal> reranking/scoring
</sect3>
<sect3 id="administration-ranking-dynamic-cql">
- <title>Dynamically ranking CQL queries</title>
+ <title>Dynamically ranking &acro.cql; queries</title>
<para>
- Dynamic ranking can be enabled during sever side CQL
+ Dynamic ranking can be enabled during sever side &acro.cql;
query expansion by adding <literal>@attr 2=102</literal>
- chunks to the CQL config file. For example
+ chunks to the &acro.cql; config file. For example
<screen>
relationModifier.relevant = 2=102
</screen>
- invokes dynamic ranking each time a CQL query of the form
+ invokes dynamic ranking each time a &acro.cql; query of the form
<screen>
Z> querytype cql
Z> f alvis.text =/relevant house
</screen>
is issued. Dynamic ranking can also be automatically used on
- specific CQL indexes by (for example) setting
+ specific &acro.cql; indexes by (for example) setting
<screen>
index.alvis.text = 1=text 2=102
</screen>
- which then invokes dynamic ranking each time a CQL query of the form
+ which then invokes dynamic ranking each time a &acro.cql; query of the form
<screen>
Z> querytype cql
Z> f alvis.text = house
<sect2 id="administration-ranking-sorting">
<title>Sorting</title>
<para>
- Zebra sorts efficiently using special sorting indexes
+ &zebra; sorts efficiently using special sorting indexes
(type=<literal>s</literal>; so each sortable index must be known
at indexing time, specified in the configuration of record
- indexing. For example, to enable sorting according to the BIB-1
+ indexing. For example, to enable sorting according to the &acro.bib1;
<literal>Date/time-added-to-db</literal> field, one could add the line
<screen>
xelm /*/@created Date/time-added-to-db:s
<para>
Indexing can be specified at searching time using a query term
carrying the non-standard
- BIB-1 attribute-type <literal>7</literal>. This removes the
- need to send a Z39.50 <literal>Sort Request</literal>
+ &acro.bib1; attribute-type <literal>7</literal>. This removes the
+ need to send a &acro.z3950; <literal>Sort Request</literal>
separately, and can dramatically improve latency when the client
and server are on separate networks.
The sorting part of the query is separate from the rest of the
</para>
<para>
A sorting subquery needs two attributes: an index (such as a
- BIB-1 type-1 attribute) specifying which index to sort on, and a
+ &acro.bib1; type-1 attribute) specifying which index to sort on, and a
type-7 attribute whose value is be <literal>1</literal> for
ascending sorting, or <literal>2</literal> for descending. The
term associated with the sorting attribute is the priority of
on.
</para>
<para>For example, a search for water, sort by title (ascending),
- is expressed by the PQF query
+ is expressed by the &acro.pqf; query
<screen>
@or @attr 1=1016 water @attr 7=1 @attr 1=4 0
</screen>
<note>
<para>
- Extended services are only supported when accessing the Zebra
- server using the <ulink url="&url.z39.50;">Z39.50</ulink>
- protocol. The <ulink url="&url.sru;">SRU</ulink> protocol does
+ Extended services are only supported when accessing the &zebra;
+ server using the <ulink url="&url.z39.50;">&acro.z3950;</ulink>
+ protocol. The <ulink url="&url.sru;">&acro.sru;</ulink> protocol does
not support extended services.
</para>
</note>
<para>
The extended services are not enabled by default in zebra - due to the
- fact that they modify the system. Zebra can be configured
+ fact that they modify the system. &zebra; can be configured
to allow anybody to
search, and to allow only updates for a particular admin user
in the main zebra configuration file <filename>zebra.cfg</filename>.
<screen>
admin:secret
</screen>
- It is essential to configure Zebra to store records internally,
+ It is essential to configure &zebra; to store records internally,
and to support
modifications and deletion of records:
<screen>
storeKeys: 1
</screen>
The general record type should be set to any record filter which
- is able to parse XML records, you may use any of the two
+ is able to parse &acro.xml; records, you may use any of the two
declarations (but not both simultaneously!)
<screen>
- recordType: grs.xml
- # recordType: alvis.filter_alvis_config.xml
+ recordType: dom.filter_dom_conf.xml
+ # recordType: grs.xml
</screen>
+ Notice the difference to the specific instructions
+ <screen>
+ recordType.xml: dom.filter_dom_conf.xml
+ # recordType.xml: grs.xml
+ </screen>
+ which only work when indexing XML files from the filesystem using
+ the <literal>*.xml</literal> naming convention.
+ </para>
+ <para>
To enable transaction safe shadow indexing,
which is extra important for this kind of operation, set
<screen>
shadow: directoryname: size (e.g. 1000M)
</screen>
+ See <xref linkend="zebra-cfg"/> for additional information on
+ these configuration options.
</para>
<note>
<para>
It is not possible to carry information about record types or
- similar to Zebra when using extended services, due to
- limitations of the <ulink url="&url.z39.50;">Z39.50</ulink>
+ similar to &zebra; when using extended services, due to
+ limitations of the <ulink url="&url.z39.50;">&acro.z3950;</ulink>
protocol. Therefore, indexing filters can not be chosen on a
- per-record basis. One and only one general XML indexing filter
+ per-record basis. One and only one general &acro.xml; indexing filter
must be defined.
<!-- but because it is represented as an OID, we would need some
form of proprietary mapping scheme between record type strings and
OIDs. -->
<!--
However, as a minimum, it would be extremely useful to enable
- people to use MARC21, assuming grs.marcxml.marc21 as a record
+ people to use &acro.marc21;, assuming grs.marcxml.marc21 as a record
type.
-->
</para>
<sect2 id="administration-extended-services-z3950">
- <title>Extended services in the Z39.50 protocol</title>
+ <title>Extended services in the &acro.z3950; protocol</title>
<para>
- The <ulink url="&url.z39.50;">Z39.50</ulink> standard allows
+ The <ulink url="&url.z39.50;">&acro.z3950;</ulink> standard allows
servers to accept special binary <emphasis>extended services</emphasis>
protocol packages, which may be used to insert, update and delete
records into servers. These carry control and update
</para>
<table id="administration-extended-services-z3950-table" frame="top">
- <title>Extended services Z39.50 Package Fields</title>
+ <title>Extended services &acro.z3950; Package Fields</title>
<tgroup cols="3">
<thead>
<row>
</row>
<row>
<entry><literal>record</literal></entry>
- <entry><literal>XML string</literal></entry>
- <entry>An XML formatted string containing the record</entry>
- </row>
- <row>
- <entry><literal>syntax</literal></entry>
- <entry><literal>'xml'</literal></entry>
- <entry>Only XML record syntax is supported</entry>
+ <entry><literal>&acro.xml; string</literal></entry>
+ <entry>An &acro.xml; formatted string containing the record</entry>
</row>
+ <row>
+ <entry><literal>syntax</literal></entry>
+ <entry><literal>'xml'</literal></entry>
+ <entry>XML/SUTRS/MARC. GRS-1 not supported.
+ The default filter (record type) as given by recordType in
+ zebra.cfg is used to parse the record.</entry>
+ </row>
<row>
<entry><literal>recordIdOpaque</literal></entry>
<entry><literal>string</literal></entry>
<entry>
- Optional client-supplied, opaque record
+ Optional client-supplied, opaque record
identifier used under insert operations.
</entry>
</row>
<row>
<entry><literal>recordIdNumber </literal></entry>
<entry><literal>positive number</literal></entry>
- <entry>Zebra's internal system number, only for update
- actions.
+ <entry>&zebra;'s internal system number,
+ not allowed for <literal>recordInsert</literal> or
+ <literal>specialUpdate</literal> actions which result in fresh
+ record inserts.
</entry>
</row>
<row>
<literal>recordDelete</literal> (will fail if the record does not
exist), and
<literal>specialUpdate</literal> (will insert or update the record
- as needed).
+ as needed, record deletion is not possible).
</para>
<para>
- During a <literal>recordInsert</literal> action, the
+ During all actions, the
usual rules for internal record ID generation apply, unless an
- optional <literal>recordIdNumber</literal> Zebra internal ID or a
+ optional <literal>recordIdNumber</literal> &zebra; internal ID or a
<literal>recordIdOpaque</literal> string identifier is assigned.
The default ID generation is
configured using the <literal>recordId:</literal> from
- <filename>zebra.cfg</filename>.
+ <filename>zebra.cfg</filename>.
+ See <xref linkend="zebra-cfg"/>.
</para>
<para>
- The actions <literal>recordReplace</literal> or
- <literal>recordDelete</literal> need specification of the additional
- <literal>recordIdNumber</literal> parameter, which must be an
- existing Zebra internal system ID number, or the optional
- <literal>recordIdOpaque</literal> string parameter.
+ Setting of the <literal>recordIdNumber</literal> parameter,
+ which must be an existing &zebra; internal system ID number, is not
+ allowed during any <literal>recordInsert</literal> or
+ <literal>specialUpdate</literal> action resulting in fresh record
+ inserts.
</para>
<para>
When retrieving existing
- records indexed with GRS indexing filters, the Zebra internal
+ records indexed with &acro.grs1; indexing filters, the &zebra; internal
ID number is returned in the field
<literal>/*/id:idzebra/localnumber</literal> in the namespace
<literal>xmlns:id="http://www.indexdata.dk/zebra/"</literal>,
where it can be picked up for later record updates or deletes.
</para>
+
<para>
- Records indexed with the <literal>alvis</literal> filter
- have similar means to discover the internal Zebra ID.
+ A new element set for retrieval of internal record
+ data has been added, which can be used to access minimal records
+ containing only the <literal>recordIdNumber</literal> &zebra;
+ internal ID, or the <literal>recordIdOpaque</literal> string
+ identifier. This works for any indexing filter used.
+ See <xref linkend="special-retrieval"/>.
</para>
-
+
<para>
The <literal>recordIdOpaque</literal> string parameter
is an client-supplied, opaque record
records. This identifier will
replace zebra's own automagic identifier generation with a unique
mapping from <literal>recordIdOpaque</literal> to the
- Zebra internal <literal>recordIdNumber</literal>.
+ &zebra; internal <literal>recordIdNumber</literal>.
<emphasis>The opaque <literal>recordIdOpaque</literal> string
identifiers
are not visible in retrieval records, nor are
searchable, so the value of this parameter is
questionable. It serves mostly as a convenient mapping from
- application domain string identifiers to Zebra internal ID's.
+ application domain string identifiers to &zebra; internal ID's.
</emphasis>
</para>
</sect2>
]]>
</screen>
Now the <literal>Default</literal> database was created,
- we can insert an XML file (esdd0006.grs
+ we can insert an &acro.xml; file (esdd0006.grs
from example/gils/records) and index it:
<screen>
<
projects / idzebra-moved-to-github.git / blobdiff