<chapter id="administration">
- <!-- $Id: administration.xml,v 1.25 2006-02-16 16:50:18 marc Exp $ -->
- <title>Administrating Zebra</title>
+ <title>Administrating &zebra;</title>
<!-- ### It's a bit daft that this chapter (which describes half of
the configuration-file formats) is separated from
"recordmodel-grs.xml" (which describes the other half) by the
-->
<para>
- Unlike many simpler retrieval systems, Zebra supports safe, incremental
+ Unlike many simpler retrieval systems, &zebra; supports safe, incremental
updates to an existing index.
</para>
<para>
- Normally, when Zebra modifies the index it reads a number of records
+ Normally, when &zebra; modifies the index it reads a number of records
that you specify.
Depending on your specifications and on the contents of each record
one the following events take place for each record:
<listitem>
<para>
The record is indexed as if it never occurred before.
- Either the Zebra system doesn't know how to identify the record or
- Zebra can identify the record but didn't find it to be already indexed.
+ Either the &zebra; system doesn't know how to identify the record or
+ &zebra; can identify the record but didn't find it to be already indexed.
</para>
</listitem>
</varlistentry>
</para>
<para>
- Please note that in both the modify- and delete- case the Zebra
+ Please note that in both the modify- and delete- case the &zebra;
indexer must be able to generate a unique key that identifies the record
in question (more on this below).
</para>
<para>
- To administrate the Zebra retrieval system, you run the
+ To administrate the &zebra; retrieval system, you run the
<literal>zebraidx</literal> program.
This program supports a number of options which are preceded by a dash,
and a few commands (not preceded by dash).
</para>
<para>
- Both the Zebra administrative tool and the Z39.50 server share a
+ Both the &zebra; administrative tool and the &acro.z3950; server share a
set of index files and a global configuration file.
The name of the configuration file defaults to
<literal>zebra.cfg</literal>.
Indexing is a per-record process, in which either insert/modify/delete
will occur. Before a record is indexed search keys are extracted from
whatever might be the layout the original record (sgml,html,text, etc..).
- The Zebra system currently supports two fundamental types of records:
+ The &zebra; system currently supports two fundamental types of records:
structured and simple text.
To specify a particular extraction process, use either the
command line option <literal>-t</literal> or specify a
</sect1>
- <sect1 id="configuration-file">
- <title>The Zebra Configuration File</title>
+ <sect1 id="zebra-cfg">
+ <title>The &zebra; Configuration File</title>
<para>
- The Zebra configuration file, read by <literal>zebraidx</literal> and
+ The &zebra; configuration file, read by <literal>zebraidx</literal> and
<literal>zebrasrv</literal> defaults to <literal>zebra.cfg</literal>
unless specified by <literal>-c</literal> option.
</para>
In the configuration file, the group name is placed before the option
name itself, separated by a dot (.). For instance, to set the record type
for group <literal>public</literal> to <literal>grs.sgml</literal>
- (the SGML-like format for structured records) you would write:
+ (the &acro.sgml;-like format for structured records) you would write:
</para>
<para>
<replaceable>database</replaceable></term>
<listitem>
<para>
- Specifies the Z39.50 database name.
+ Specifies the &acro.z3950; database name.
<!-- FIXME - now we can have multiple databases in one server. -H -->
</para>
</listitem>
<listitem>
<para>
Specifies whether the records should be stored internally
- in the Zebra system files.
+ in the &zebra; system files.
If you want to maintain the raw records yourself,
this option should be false (0).
- If you want Zebra to take care of the records for you, it
+ If you want &zebra; to take care of the records for you, it
should be true(1).
</para>
</listitem>
<term>register: <replaceable>register-location</replaceable></term>
<listitem>
<para>
- Specifies the location of the various register files that Zebra uses
+ Specifies the location of the various register files that &zebra; uses
to represent your databases.
See <xref linkend="register-location"/>.
</para>
<term>shadow: <replaceable>register-location</replaceable></term>
<listitem>
<para>
- Enables the <emphasis>safe update</emphasis> facility of Zebra, and
+ Enables the <emphasis>safe update</emphasis> facility of &zebra;, and
tells the system where to place the required, temporary files.
See <xref linkend="shadow-registers"/>.
</para>
<para>
Specifies a path of profile specification files.
The path is composed of one or more directories separated by
- colon. Similar to PATH for UNIX systems.
+ colon. Similar to <literal>PATH</literal> for UNIX systems.
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>modulePath: <replaceable>path</replaceable></term>
+ <listitem>
+ <para>
+ Specifies a path of record filter modules.
+ The path is composed of one or more directories separated by
+ colon. Similar to <literal>PATH</literal> for UNIX systems.
+ The 'make install' procedure typically puts modules in
+ <filename>/usr/local/lib/idzebra-2.0/modules</filename>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>index: <replaceable>filename</replaceable></term>
+ <listitem>
+ <para>
+ Defines the filename which holds fields structure
+ definitions. If omitted, the file <filename>default.idx</filename>
+ is read.
+ Refer to <xref linkend="default-idx-file"/> for
+ more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sortmax: <replaceable>integer</replaceable></term>
+ <listitem>
+ <para>
+ Specifies the maximum number of records that will be sorted
+ in a result set. If the result set contains more than
+ <replaceable>integer</replaceable> records, records after the
+ limit will not be sorted. If omitted, the default value is
+ 1,000.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>staticrank: <replaceable>integer</replaceable></term>
+ <listitem>
+ <para>
+ Enables whether static ranking is to be enabled (1) or
+ disabled (0). If omitted, it is disabled - corresponding
+ to a value of 0.
+ Refer to <xref linkend="administration-ranking-static"/> .
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry>
+ <term>estimatehits: <replaceable>integer</replaceable></term>
+ <listitem>
+ <para>
+ Controls whether &zebra; should calculate approximate hit counts and
+ at which hit count it is to be enabled.
+ A value of 0 disables approximate hit counts.
+ For a positive value approximate hit count is enabled
+ if it is known to be larger than <replaceable>integer</replaceable>.
+ </para>
+ <para>
+ Approximate hit counts can also be triggered by a particular
+ attribute in a query.
+ Refer to <xref linkend="querymodel-zebra-global-attr-limit"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term>attset: <replaceable>filename</replaceable></term>
<listitem>
<para>
- Specifies the filename(s) of attribute set files for use in
- searching. At least the Bib-1 set should be loaded
- (<literal>bib1.att</literal>).
- The <literal>profilePath</literal> setting is used to look for
- the specified files.
- See <xref linkend="attset-files"/>
+ Specifies the filename(s) of attribute set files for use in
+ searching. In many configurations <filename>bib1.att</filename>
+ is used, but that is not required. If Classic Explain
+ attributes is to be used for searching,
+ <filename>explain.att</filename> must be given.
+ The path to att-files in general can be given using
+ <literal>profilePath</literal> setting.
+ See also <xref linkend="attset-files"/>.
</para>
</listitem>
</varlistentry>
<term>root: <replaceable>dir</replaceable></term>
<listitem>
<para>
- Specifies a directory base for Zebra. All relative paths
+ Specifies a directory base for &zebra;. All relative paths
given (in profilePath, register, shadow) are based on this
- directory. This setting is useful if your Zebra server
+ directory. This setting is useful if your &zebra; server
is running in a different directory from where
<literal>zebra.cfg</literal> is located.
</para>
<term>passwd: <replaceable>file</replaceable></term>
<listitem>
<para>
- Specifies a file with description of user accounts for Zebra.
+ Specifies a file with description of user accounts for &zebra;.
The format is similar to that known to Apache's htpasswd files
and UNIX' passwd files. Non-empty lines not beginning with
# are considered account lines. There is one account per-line.
<term>passwd.c: <replaceable>file</replaceable></term>
<listitem>
<para>
- Specifies a file with description of user accounts for Zebra.
+ Specifies a file with description of user accounts for &zebra;.
File format is similar to that used by the passwd directive except
that the password are encrypted. Use Apache's htpasswd or similar
- for maintenanace.
+ for maintenance.
</para>
</listitem>
</varlistentry>
<replaceable>permstring</replaceable></term>
<listitem>
<para>
- Specifies permissions (priviledge) for a user that are allowed
- to access Zebra via the passwd system. There are two kinds
+ Specifies permissions (privilege) for a user that are allowed
+ to access &zebra; via the passwd system. There are two kinds
of permissions currently: read (r) and write(w). By default
users not listed in a permission directive are given the read
- priviledge. To specify permissions for a user with no
- username, or Z39.50 anonymous style use
+ privilege. To specify permissions for a user with no
+ username, or &acro.z3950; anonymous style use
<literal>anonymous</literal>. The permstring consists of
a sequence of characters. Include character <literal>w</literal>
- for write/update access, <literal>r</literal> for read access.
+ for write/update access, <literal>r</literal> for read access and
+ <literal>a</literal> to allow anonymous access through this account.
</para>
</listitem>
</varlistentry>
<varlistentry>
- <term>dbaccess <replaceable>accessfile</replaceable></term>
+ <term>dbaccess: <replaceable>accessfile</replaceable></term>
<listitem>
<para>
Names a file which lists database subscriptions for individual users.
- The access file should consists of lines of the form <literal>username:
- dbnames</literal>, where dbnames is a list of database names, seprated by
- '+'. No whitespace is allowed in the database list.
+ The access file should consists of lines of the form
+ <literal>username: dbnames</literal>, where dbnames is a list of
+ database names, separated by '+'. No whitespace is allowed in the
+ database list.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>encoding: <replaceable>charsetname</replaceable></term>
+ <listitem>
+ <para>
+ Tells &zebra; to interpret the terms in Z39.50 queries as
+ having been encoded using the specified character
+ encoding. The default is <literal>ISO-8859-1</literal>; one
+ useful alternative is <literal>UTF-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>storeKeys: <replaceable>value</replaceable></term>
+ <listitem>
+ <para>
+ Specifies whether &zebra; keeps a copy of indexed keys.
+ Use a value of 1 to enable; 0 to disable. If storeKeys setting is
+ omitted, it is enabled. Enabled storeKeys
+ are required for updating and deleting records. Disable only
+ storeKeys to save space and only plan to index data once.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>storeData: <replaceable>value</replaceable></term>
+ <listitem>
+ <para>
+ Specifies whether &zebra; keeps a copy of indexed records.
+ Use a value of 1 to enable; 0 to disable. If storeData setting is
+ omitted, it is enabled. A storeData setting of 0 (disabled) makes
+ Zebra fetch records from the original locaction in the file
+ system using filename, file offset and file length. For the
+ DOM and ALVIS filter, the storeData setting is ignored.
</para>
</listitem>
</varlistentry>
<title>Locating Records</title>
<para>
- The default behavior of the Zebra system is to reference the
+ The default behavior of the &zebra; system is to reference the
records from their original location, i.e. where they were found when you
- ran <literal>zebraidx</literal>.
+ run <literal>zebraidx</literal>.
That is, when a client wishes to retrieve a record
following a search operation, the files are accessed from the place
where you originally put them - if you remove the files (without
If your input files are not permanent - for example if you retrieve
your records from an outside source, or if they were temporarily
mounted on a CD-ROM drive,
- you may want Zebra to make an internal copy of them. To do this,
+ you may want &zebra; to make an internal copy of them. To do this,
you specify 1 (true) in the <literal>storeData</literal> setting. When
- the Z39.50 server retrieves the records they will be read from the
+ the &acro.z3950; server retrieves the records they will be read from the
internal file structures of the system.
</para>
<para>
Consider a system in which you have a group of text files called
<literal>simple</literal>.
- That group of records should belong to a Z39.50 database called
+ That group of records should belong to a &acro.z3950; database called
<literal>textbase</literal>.
The following <literal>zebra.cfg</literal> file will suffice:
</para>
To enable indexing with pathname IDs, you must specify
<literal>file</literal> as the value of <literal>recordId</literal>
in the configuration file. In addition, you should set
- <literal>storeKeys</literal> to <literal>1</literal>, since the Zebra
+ <literal>storeKeys</literal> to <literal>1</literal>, since the &zebra;
indexer must save additional information about the contents of each record
in order to modify the indexes correctly at a later time.
</para>
<note>
<para>You cannot start out with a group of records with simple
indexing (no record IDs as in the previous section) and then later
- enable file record Ids. Zebra must know from the first time that you
+ enable file record Ids. &zebra; must know from the first time that you
index the group that
the files should be indexed with file record IDs.
</para>
information. If you have a group of records that explicitly associates
an ID with each record, this method is convenient. For example, the
record format may contain a title or a ID-number - unique within the group.
- In either case you specify the Z39.50 attribute set and use-attribute
+ In either case you specify the &acro.z3950; attribute set and use-attribute
location in which this information is stored, and the system looks at
that field to determine the identity of the record.
</para>
</para>
<para>
- For instance, the sample GILS records that come with the Zebra
+ For instance, the sample GILS records that come with the &zebra;
distribution contain a unique ID in the data tagged Control-Identifier.
- The data is mapped to the Bib-1 use attribute Identifier-standard
+ The data is mapped to the &acro.bib1; use attribute Identifier-standard
(code 1007). To use this field as a record id, specify
<literal>(bib1,Identifier-standard)</literal> as the value of the
<literal>recordId</literal> in the configuration file.
</para>
<para>
- (see <xref linkend="record-model-grs"/>
+ (see <xref linkend="grs"/>
for details of how the mapping between elements of your records and
searchable attributes is established).
</para>
<literal>zebraidx</literal>. If you wish to store these, possibly large,
files somewhere else, you must add the <literal>register</literal>
entry to the <literal>zebra.cfg</literal> file.
- Furthermore, the Zebra system allows its file
+ Furthermore, the &zebra; system allows its file
structures to span multiple file systems, which is useful for
managing very large databases.
</para>
The value of the <literal>register</literal> setting is a sequence
of tokens. Each token takes the form:
- <screen>
- <emphasis>dir</emphasis><literal>:</literal><emphasis>size</emphasis>.
- </screen>
+ <emphasis>dir</emphasis><literal>:</literal><emphasis>size</emphasis>
The <emphasis>dir</emphasis> specifies a directory in which index files
will be stored and the <emphasis>size</emphasis> specifies the maximum
- size of all files in that directory. The Zebra indexer system fills
+ size of all files in that directory. The &zebra; indexer system fills
each directory in the order specified and use the next specified
directories as needed.
The <emphasis>size</emphasis> is an integer followed by a qualifier
<literal>k</literal> for kilobytes.
<literal>M</literal> for megabytes,
<literal>G</literal> for gigabytes.
+ Specifying a negative value disables the checking (it still needs the unit,
+ use <literal>-1b</literal>).
</para>
<para>
- For instance, if you have allocated two disks for your register, and
+ For instance, if you have allocated three disks for your register, and
the first disk is mounted
- on <literal>/d1</literal> and has 2GB of free space and the
- second, mounted on <literal>/d2</literal> has 3.6 GB, you could
- put this entry in your configuration file:
+ on <literal>/d1</literal> and has 2GB of free space, the
+ second, mounted on <literal>/d2</literal> has 3.6 GB, and the third,
+ on which you have more space than you bother to worry about, mounted on
+ <literal>/d3</literal> you could put this entry in your configuration file:
<screen>
- register: /d1:2G /d2:3600M
+ register: /d1:2G /d2:3600M /d3:-1b
</screen>
-
</para>
<para>
- Note that Zebra does not verify that the amount of space specified is
+ Note that &zebra; does not verify that the amount of space specified is
actually available on the directory (file system) specified - it is
your responsibility to ensure that enough space is available, and that
other applications do not attempt to use the free space. In a large
production system, it is recommended that you allocate one or more
- file system exclusively to the Zebra register files.
+ file system exclusively to the &zebra; register files.
</para>
</sect1>
<sect1 id="shadow-registers">
<title>Safe Updating - Using Shadow Registers</title>
- <sect2>
+ <sect2 id="shadow-registers-description">
<title>Description</title>
<para>
- The Zebra server supports <emphasis>updating</emphasis> of the index
+ The &zebra; server supports <emphasis>updating</emphasis> of the index
structures. That is, you can add, modify, or remove records from
- databases managed by Zebra without rebuilding the entire index.
+ databases managed by &zebra; without rebuilding the entire index.
Since this process involves modifying structured files with various
references between blocks of data in the files, the update process
is inherently sensitive to system crashes, or to process interruptions:
<para>
You can solve these problems by enabling the shadow register system in
- Zebra.
+ &zebra;.
During the updating procedure, <literal>zebraidx</literal> will temporarily
write changes to the involved files in a set of "shadow
files", without modifying the files that are accessed by the
</sect2>
- <sect2>
+ <sect2 id="shadow-registers-how-to-use">
<title>How to Use Shadow Register Files</title>
<para>
<sect1 id="administration-ranking">
- <title>Static and Dynamic Ranking</title>
+ <title>Relevance Ranking and Sorting of Result Sets</title>
+
+ <sect2 id="administration-overview">
+ <title>Overview</title>
+ <para>
+ The default ordering of a result set is left up to the server,
+ which inside &zebra; means sorting in ascending document ID order.
+ This is not always the order humans want to browse the sometimes
+ quite large hit sets. Ranking and sorting comes to the rescue.
+ </para>
+
+ <para>
+ In cases where a good presentation ordering can be computed at
+ indexing time, we can use a fixed <literal>static ranking</literal>
+ scheme, which is provided for the <literal>alvis</literal>
+ indexing filter. This defines a fixed ordering of hit lists,
+ independently of the query issued.
+ </para>
+
+ <para>
+ There are cases, however, where relevance of hit set documents is
+ highly dependent on the query processed.
+ Simply put, <literal>dynamic relevance ranking</literal>
+ sorts a set of retrieved records such that those most likely to be
+ relevant to your request are retrieved first.
+ Internally, &zebra; retrieves all documents that satisfy your
+ query, and re-orders the hit list to arrange them based on
+ a measurement of similarity between your query and the content of
+ each record.
+ </para>
+
+ <para>
+ Finally, there are situations where hit sets of documents should be
+ <literal>sorted</literal> during query time according to the
+ lexicographical ordering of certain sort indexes created at
+ indexing time.
+ </para>
+ </sect2>
+
+
+ <sect2 id="administration-ranking-static">
+ <title>Static Ranking</title>
<para>
- Zebra uses internally inverted indexes to look up term occurencies
+ &zebra; uses internally inverted indexes to look up term frequencies
in documents. Multiple queries from different indexes can be
combined by the binary boolean operations <literal>AND</literal>,
<literal>OR</literal> and/or <literal>NOT</literal> (which
<screen>
staticrank: 1
</screen>
- directive in the main core Zebra config file, the internal document
- keys used for ordering are augmented by a preceeding integer, which
+ directive in the main core &zebra; configuration file, the internal document
+ keys used for ordering are augmented by a preceding integer, which
contains the static rank of a given document, and the index lists
are ordered
first by ascending static rank,
then by ascending document <literal>ID</literal>.
- </para>
- <para>
- This implies that the default rank <literal>0</literal>
- is the best rank at the
- beginning of the list, and <literal>max int</literal>
- is the worst static rank.
+ Zero
+ is the ``best'' rank, as it occurs at the
+ beginning of the list; higher numbers represent worse scores.
</para>
<para>
The experimental <literal>alvis</literal> filter provides a
- directive to fetch static rank information out of the indexed XML
- records, thus making <emphasis>all</emphasis> hit sets orderd
+ directive to fetch static rank information out of the indexed &acro.xml;
+ records, thus making <emphasis>all</emphasis> hit sets ordered
after <emphasis>ascending</emphasis> static
rank, and for those doc's which have the same static rank, ordered
after <emphasis>ascending</emphasis> doc <literal>ID</literal>.
- See <xref linkend="record-model-alvisxslt"/> for the glory details.
+ See <xref linkend="record-model-alvisxslt"/> for the gory details.
</para>
+ </sect2>
+
+
+ <sect2 id="administration-ranking-dynamic">
+ <title>Dynamic Ranking</title>
<para>
- If one wants to do a little fiddeling with the static rank order,
- one has to invoke additional re-ranking/re-ordering using dynamic
- reranking or score functions. These functions return positive
- interger scores, where <emphasis>highest</emphasis> score is
- <emphasis>best</emphasis>, which means that the
- hit sets will be sorted according to
- <emphasis>decending</emphasis>
+ In order to fiddle with the static rank order, it is necessary to
+ invoke additional re-ranking/re-ordering using dynamic
+ ranking or score functions. These functions return positive
+ integer scores, where <emphasis>highest</emphasis> score is
+ ``best'';
+ hit sets are sorted according to <emphasis>descending</emphasis>
scores (in contrary
to the index lists which are sorted according to
- <emphasis>ascending</emphasis> rank number and document ID).
+ ascending rank number and document ID).
</para>
- <!--
<para>
- Those are defined in the zebra C source files
- <screen>
- "rank-1" : zebra/index/rank1.c
- default TF/IDF like zebra dynamic ranking
- "rank-static" : zebra/index/rankstatic.c
- do-nothing dummy static ranking (this is just to prove
- that the static rank can be used in dynamic ranking functions)
- "zvrank" : zebra/index/zvrank.c
- many different dynamic TF/IDF ranking functions
- </screen>
+ Dynamic ranking is enabled by a directive like one of the
+ following in the zebra configuration file (use only one of these a time!):
+ <screen>
+ rank: rank-1 # default TDF-IDF like
+ rank: rank-static # dummy do-nothing
+ </screen>
</para>
- -->
+
<para>
- Those are in the zebra config file enabled by a directive like (use
- only one of these a time!):
- <screen>
- rank: rank-1 # default
- rank: rank-static # dummy
- rank: zvrank # TDF-IDF like
+ Dynamic ranking is done at query time rather than
+ indexing time (this is why we
+ call it ``dynamic ranking'' in the first place ...)
+ It is invoked by adding
+ the &acro.bib1; relation attribute with
+ value ``relevance'' to the &acro.pqf; query (that is,
+ <literal>@attr 2=102</literal>, see also
+ <ulink url="&url.z39.50;bib1.html">
+ The &acro.bib1; Attribute Set Semantics</ulink>, also in
+ <ulink url="&url.z39.50.attset.bib1;">HTML</ulink>).
+ To find all articles with the word <literal>Eoraptor</literal> in
+ the title, and present them relevance ranked, issue the &acro.pqf; query:
+ <screen>
+ @attr 2=102 @attr 1=4 Eoraptor
</screen>
- Notice that the <literal>rank-1</literal> and
- <literal>zvrank</literal> do not use the static rank
- information in the list keys, and will produce the same ordering
- with our without static ranking enabled.
</para>
+
+ <sect3 id="administration-ranking-dynamic-rank1">
+ <title>Dynamically ranking using &acro.pqf; queries with the 'rank-1'
+ algorithm</title>
+
<para>
+ The default <literal>rank-1</literal> ranking module implements a
+ TF/IDF (Term Frequecy over Inverse Document Frequency) like
+ algorithm. In contrast to the usual definition of TF/IDF
+ algorithms, which only considers searching in one full-text
+ index, this one works on multiple indexes at the same time.
+ More precisely,
+ &zebra; does boolean queries and searches in specific addressed
+ indexes (there are inverted indexes pointing from terms in the
+ dictionary to documents and term positions inside documents).
+ It works like this:
+ <variablelist>
+ <varlistentry>
+ <term>Query Components</term>
+ <listitem>
+ <para>
+ First, the boolean query is dismantled into its principal components,
+ i.e. atomic queries where one term is looked up in one index.
+ For example, the query
+ <screen>
+ @attr 2=102 @and @attr 1=1010 Utah @attr 1=1018 Springer
+ </screen>
+ is a boolean AND between the atomic parts
+ <screen>
+ @attr 2=102 @attr 1=1010 Utah
+ </screen>
+ and
+ <screen>
+ @attr 2=102 @attr 1=1018 Springer
+ </screen>
+ which gets processed each for itself.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Atomic hit lists</term>
+ <listitem>
+ <para>
+ Second, for each atomic query, the hit list of documents is
+ computed.
+ </para>
+ <para>
+ In this example, two hit lists for each index
+ <literal>@attr 1=1010</literal> and
+ <literal>@attr 1=1018</literal> are computed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Atomic scores</term>
+ <listitem>
+ <para>
+ Third, each document in the hit list is assigned a score (_if_ ranking
+ is enabled and requested in the query) using a TF/IDF scheme.
+ </para>
+ <para>
+ In this example, both atomic parts of the query assign the magic
+ <literal>@attr 2=102</literal> relevance attribute, and are
+ to be used in the relevance ranking functions.
+ </para>
+ <para>
+ It is possible to apply dynamic ranking on only parts of the
+ &acro.pqf; query:
+ <screen>
+ @and @attr 2=102 @attr 1=1010 Utah @attr 1=1018 Springer
+ </screen>
+ searches for all documents which have the term 'Utah' on the
+ body of text, and which have the term 'Springer' in the publisher
+ field, and sort them in the order of the relevance ranking made on
+ the body-of-text index only.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Hit list merging</term>
+ <listitem>
+ <para>
+ Fourth, the atomic hit lists are merged according to the boolean
+ conditions to a final hit list of documents to be returned.
+ </para>
+ <para>
+ This step is always performed, independently of the fact that
+ dynamic ranking is enabled or not.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Document score computation</term>
+ <listitem>
+ <para>
+ Fifth, the total score of a document is computed as a linear
+ combination of the atomic scores of the atomic hit lists
+ </para>
+ <para>
+ Ranking weights may be used to pass a value to a ranking
+ algorithm, using the non-standard &acro.bib1; attribute type 9.
+ This allows one branch of a query to use one value while
+ another branch uses a different one. For example, we can search
+ for <literal>utah</literal> in the
+ <literal>@attr 1=4</literal> index with weight 30, as
+ well as in the <literal>@attr 1=1010</literal> index with weight 20:
+ <screen>
+ @attr 2=102 @or @attr 9=30 @attr 1=4 utah @attr 9=20 @attr 1=1010 city
+ </screen>
+ </para>
+ <para>
+ The default weight is
+ sqrt(1000) ~ 34 , as the &acro.z3950; standard prescribes that the top score
+ is 1000 and the bottom score is 0, encoded in integers.
+ </para>
+ <warning>
+ <para>
+ The ranking-weight feature is experimental. It may change in future
+ releases of zebra.
+ </para>
+ </warning>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Re-sorting of hit list</term>
+ <listitem>
+ <para>
+ Finally, the final hit list is re-ordered according to scores.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+
+<!--
+Still need to describe the exact TF/IDF formula. Here's the info, need -->
+<!--to extract it in human readable form .. MC
+
+static int calc (void *set_handle, zint sysno, zint staticrank,
+ int *stop_flag)
+{
+ int i, lo, divisor, score = 0;
+ struct rank_set_info *si = (struct rank_set_info *) set_handle;
+
+ if (!si->no_rank_entries)
+ return -1; /* ranking not enabled for any terms */
+
+ for (i = 0; i < si->no_entries; i++)
+ {
+ yaz_log(log_level, "calc: i=%d rank_flag=%d lo=%d",
+ i, si->entries[i].rank_flag, si->entries[i].local_occur);
+ if (si->entries[i].rank_flag && (lo = si->entries[i].local_occur))
+ score += (8+log2_int (lo)) * si->entries[i].global_inv *
+ si->entries[i].rank_weight;
+ }
+ divisor = si->no_rank_entries * (8+log2_int (si->last_pos/si->no_entries));
+ score = score / divisor;
+ yaz_log(log_level, "calc sysno=" ZINT_FORMAT " score=%d", sysno, score);
+ if (score > 1000)
+ score = 1000;
+ /* reset the counts for the next term */
+ for (i = 0; i < si->no_entries; i++)
+ si->entries[i].local_occur = 0;
+ return score;
+}
+
+
+where lo = si->entries[i].local_occur is the local documents term-within-index frequency, si->entries[i].global_inv represents the IDF part (computed in static void *begin()), and
+si->entries[i].rank_weight is the weight assigner per index (default 34, or set in the @attr 9=xyz magic)
+
+Finally, the IDF part is computed as:
+
+static void *begin (struct zebra_register *reg,
+ void *class_handle, RSET rset, NMEM nmem,
+ TERMID *terms, int numterms)
+{
+ struct rank_set_info *si =
+ (struct rank_set_info *) nmem_malloc (nmem,sizeof(*si));
+ int i;
+
+ yaz_log(log_level, "rank-1 begin");
+ si->no_entries = numterms;
+ si->no_rank_entries = 0;
+ si->nmem=nmem;
+ si->entries = (struct rank_term_info *)
+ nmem_malloc (si->nmem, sizeof(*si->entries)*numterms);
+ for (i = 0; i < numterms; i++)
+ {
+ zint g = rset_count(terms[i]->rset);
+ yaz_log(log_level, "i=%d flags=%s '%s'", i,
+ terms[i]->flags, terms[i]->name );
+ if (!strncmp (terms[i]->flags, "rank,", 5))
+ {
+ const char *cp = strstr(terms[i]->flags+4, ",w=");
+ si->entries[i].rank_flag = 1;
+ if (cp)
+ si->entries[i].rank_weight = atoi (cp+3);
+ else
+ si->entries[i].rank_weight = 34; /* sqrroot of 1000 */
+ yaz_log(log_level, " i=%d weight=%d g="ZINT_FORMAT, i,
+ si->entries[i].rank_weight, g);
+ (si->no_rank_entries)++;
+ }
+ else
+ si->entries[i].rank_flag = 0;
+ si->entries[i].local_occur = 0; /* FIXME */
+ si->entries[i].global_occur = g;
+ si->entries[i].global_inv = 32 - log2_int (g);
+ yaz_log(log_level, " global_inv = %d g = " ZINT_FORMAT,
+ (int) (32-log2_int (g)), g);
+ si->entries[i].term = terms[i];
+ si->entries[i].term_index=i;
+ terms[i]->rankpriv = &(si->entries[i]);
+ }
+ return si;
+}
+
+
+where g = rset_count(terms[i]->rset) is the count of all documents in this specific index hit list, and the IDF part then is
+
+ si->entries[i].global_inv = 32 - log2_int (g);
+ -->
+
+ </para>
+
+
+ <para>
+ The <literal>rank-1</literal> algorithm
+ does not use the static rank
+ information in the list keys, and will produce the same ordering
+ with or without static ranking enabled.
+ </para>
+
+
+ <!--
+ <sect3 id="administration-ranking-dynamic-rank1">
+ <title>Dynamically ranking &acro.pqf; queries with the 'rank-static'
+ algorithm</title>
+ <para>
The dummy <literal>rank-static</literal> reranking/scoring
function returns just
<literal>score = max int - staticrank</literal>
- in order to preserve the ordering of hit sets with and without it's
- call.
- Obviously, to combine static and dynamic ranking usefully, one wants
+ in order to preserve the static ordering of hit sets that would
+ have been produced had it not been invoked.
+ Obviously, to combine static and dynamic ranking usefully,
+ it is necessary
to make a new ranking
- function, which is left
+ function; this is left
as an exercise for the reader.
</para>
-
+ </sect3>
+ -->
+
+ <warning>
+ <para>
+ <literal>Dynamic ranking</literal> is not compatible
+ with <literal>estimated hit sizes</literal>, as all documents in
+ a hit set must be accessed to compute the correct placing in a
+ ranking sorted list. Therefore the use attribute setting
+ <literal>@attr 2=102</literal> clashes with
+ <literal>@attr 9=integer</literal>.
+ </para>
+ </warning>
+
+ <!--
+ we might want to add ranking like this:
+ UNPUBLISHED:
+ Simple BM25 Extension to Multiple Weighted Fields
+ Stephen Robertson, Hugo Zaragoza and Michael Taylor
+ Microsoft Research
+ ser@microsoft.com
+ hugoz@microsoft.com
+ mitaylor2microsoft.com
+ -->
+
+ </sect3>
+
+ <sect3 id="administration-ranking-dynamic-cql">
+ <title>Dynamically ranking &acro.cql; queries</title>
+ <para>
+ Dynamic ranking can be enabled during sever side &acro.cql;
+ query expansion by adding <literal>@attr 2=102</literal>
+ chunks to the &acro.cql; config file. For example
+ <screen>
+ relationModifier.relevant = 2=102
+ </screen>
+ invokes dynamic ranking each time a &acro.cql; query of the form
+ <screen>
+ Z> querytype cql
+ Z> f alvis.text =/relevant house
+ </screen>
+ is issued. Dynamic ranking can also be automatically used on
+ specific &acro.cql; indexes by (for example) setting
+ <screen>
+ index.alvis.text = 1=text 2=102
+ </screen>
+ which then invokes dynamic ranking each time a &acro.cql; query of the form
+ <screen>
+ Z> querytype cql
+ Z> f alvis.text = house
+ </screen>
+ is issued.
+ </para>
+
+ </sect3>
+
+ </sect2>
+
+
+ <sect2 id="administration-ranking-sorting">
+ <title>Sorting</title>
+ <para>
+ &zebra; sorts efficiently using special sorting indexes
+ (type=<literal>s</literal>; so each sortable index must be known
+ at indexing time, specified in the configuration of record
+ indexing. For example, to enable sorting according to the &acro.bib1;
+ <literal>Date/time-added-to-db</literal> field, one could add the line
+ <screen>
+ xelm /*/@created Date/time-added-to-db:s
+ </screen>
+ to any <literal>.abs</literal> record-indexing configuration file.
+ Similarly, one could add an indexing element of the form
+ <screen><![CDATA[
+ <z:index name="date-modified" type="s">
+ <xsl:value-of select="some/xpath"/>
+ </z:index>
+ ]]></screen>
+ to any <literal>alvis</literal>-filter indexing stylesheet.
+ </para>
+ <para>
+ Indexing can be specified at searching time using a query term
+ carrying the non-standard
+ &acro.bib1; attribute-type <literal>7</literal>. This removes the
+ need to send a &acro.z3950; <literal>Sort Request</literal>
+ separately, and can dramatically improve latency when the client
+ and server are on separate networks.
+ The sorting part of the query is separate from the rest of the
+ query - the actual search specification - and must be combined
+ with it using OR.
+ </para>
+ <para>
+ A sorting subquery needs two attributes: an index (such as a
+ &acro.bib1; type-1 attribute) specifying which index to sort on, and a
+ type-7 attribute whose value is be <literal>1</literal> for
+ ascending sorting, or <literal>2</literal> for descending. The
+ term associated with the sorting attribute is the priority of
+ the sort key, where <literal>0</literal> specifies the primary
+ sort key, <literal>1</literal> the secondary sort key, and so
+ on.
+ </para>
+ <para>For example, a search for water, sort by title (ascending),
+ is expressed by the &acro.pqf; query
+ <screen>
+ @or @attr 1=1016 water @attr 7=1 @attr 1=4 0
+ </screen>
+ whereas a search for water, sort by title ascending,
+ then date descending would be
+ <screen>
+ @or @or @attr 1=1016 water @attr 7=1 @attr 1=4 0 @attr 7=2 @attr 1=30 1
+ </screen>
+ </para>
+ <para>
+ Notice the fundamental differences between <literal>dynamic
+ ranking</literal> and <literal>sorting</literal>: there can be
+ only one ranking function defined and configured; but multiple
+ sorting indexes can be specified dynamically at search
+ time. Ranking does not need to use specific indexes, so
+ dynamic ranking can be enabled and disabled without
+ re-indexing; whereas, sorting indexes need to be
+ defined before indexing.
+ </para>
+
+ </sect2>
+
+
</sect1>
<sect1 id="administration-extended-services">
<title>Extended Services: Remote Insert, Update and Delete</title>
+ <note>
+ <para>
+ Extended services are only supported when accessing the &zebra;
+ server using the <ulink url="&url.z39.50;">&acro.z3950;</ulink>
+ protocol. The <ulink url="&url.sru;">&acro.sru;</ulink> protocol does
+ not support extended services.
+ </para>
+ </note>
+
<para>
The extended services are not enabled by default in zebra - due to the
- fact that they modify the system.
- In order to allow anybody to update, use
- <screen>
- perm.anonymous: rw
- </screen>
+ fact that they modify the system. &zebra; can be configured
+ to allow anybody to
+ search, and to allow only updates for a particular admin user
in the main zebra configuration file <filename>zebra.cfg</filename>.
- Or, even better, allow only updates for a particular admin user. For
- user <literal>admin</literal>, you could use:
+ For user <literal>admin</literal>, you could use:
<screen>
+ perm.anonymous: r
perm.admin: rw
passwd: passwordfile
</screen>
- And in <filename>passwordfile</filename>, specify users and
- passwords as colon seperated strings:
+ And in the password file
+ <filename>passwordfile</filename>, you have to specify users and
+ encrypted passwords as colon separated strings.
+ Use a tool like <filename>htpasswd</filename>
+ to maintain the encrypted passwords.
<screen>
admin:secret
+ </screen>
+ It is essential to configure &zebra; to store records internally,
+ and to support
+ modifications and deletion of records:
+ <screen>
+ storeData: 1
+ storeKeys: 1
+ </screen>
+ The general record type should be set to any record filter which
+ is able to parse &acro.xml; records, you may use any of the two
+ declarations (but not both simultaneously!)
+ <screen>
+ recordType: dom.filter_dom_conf.xml
+ # recordType: grs.xml
+ </screen>
+ Notice the difference to the specific instructions
+ <screen>
+ recordType.xml: dom.filter_dom_conf.xml
+ # recordType.xml: grs.xml
</screen>
+ which only work when indexing XML files from the filesystem using
+ the <literal>*.xml</literal> naming convention.
+ </para>
+ <para>
+ To enable transaction safe shadow indexing,
+ which is extra important for this kind of operation, set
+ <screen>
+ shadow: directoryname: size (e.g. 1000M)
+ </screen>
+ See <xref linkend="zebra-cfg"/> for additional information on
+ these configuration options.
+ </para>
+ <note>
+ <para>
+ It is not possible to carry information about record types or
+ similar to &zebra; when using extended services, due to
+ limitations of the <ulink url="&url.z39.50;">&acro.z3950;</ulink>
+ protocol. Therefore, indexing filters can not be chosen on a
+ per-record basis. One and only one general &acro.xml; indexing filter
+ must be defined.
+ <!-- but because it is represented as an OID, we would need some
+ form of proprietary mapping scheme between record type strings and
+ OIDs. -->
+ <!--
+ However, as a minimum, it would be extremely useful to enable
+ people to use &acro.marc21;, assuming grs.marcxml.marc21 as a record
+ type.
+ -->
+ </para>
+ </note>
+
+
+ <sect2 id="administration-extended-services-z3950">
+ <title>Extended services in the &acro.z3950; protocol</title>
+
+ <para>
+ The <ulink url="&url.z39.50;">&acro.z3950;</ulink> standard allows
+ servers to accept special binary <emphasis>extended services</emphasis>
+ protocol packages, which may be used to insert, update and delete
+ records into servers. These carry control and update
+ information to the servers, which are encoded in seven package fields:
+ </para>
+
+ <table id="administration-extended-services-z3950-table" frame="top">
+ <title>Extended services &acro.z3950; Package Fields</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Parameter</entry>
+ <entry>Value</entry>
+ <entry>Notes</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>type</literal></entry>
+ <entry><literal>'update'</literal></entry>
+ <entry>Must be set to trigger extended services</entry>
+ </row>
+ <row>
+ <entry><literal>action</literal></entry>
+ <entry><literal>string</literal></entry>
+ <entry>
+ Extended service action type with
+ one of four possible values: <literal>recordInsert</literal>,
+ <literal>recordReplace</literal>,
+ <literal>recordDelete</literal>,
+ and <literal>specialUpdate</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>record</literal></entry>
+ <entry><literal>&acro.xml; string</literal></entry>
+ <entry>An &acro.xml; formatted string containing the record</entry>
+ </row>
+ <row>
+ <entry><literal>syntax</literal></entry>
+ <entry><literal>'xml'</literal></entry>
+ <entry>XML/SUTRS/MARC. GRS-1 not supported.
+ The default filter (record type) as given by recordType in
+ zebra.cfg is used to parse the record.</entry>
+ </row>
+ <row>
+ <entry><literal>recordIdOpaque</literal></entry>
+ <entry><literal>string</literal></entry>
+ <entry>
+ Optional client-supplied, opaque record
+ identifier used under insert operations.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>recordIdNumber </literal></entry>
+ <entry><literal>positive number</literal></entry>
+ <entry>&zebra;'s internal system number,
+ not allowed for <literal>recordInsert</literal> or
+ <literal>specialUpdate</literal> actions which result in fresh
+ record inserts.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>databaseName</literal></entry>
+ <entry><literal>database identifier</literal></entry>
+ <entry>
+ The name of the database to which the extended services should be
+ applied.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+
+ <para>
+ The <literal>action</literal> parameter can be any of
+ <literal>recordInsert</literal> (will fail if the record already exists),
+ <literal>recordReplace</literal> (will fail if the record does not exist),
+ <literal>recordDelete</literal> (will fail if the record does not
+ exist), and
+ <literal>specialUpdate</literal> (will insert or update the record
+ as needed, record deletion is not possible).
</para>
+
+ <para>
+ During all actions, the
+ usual rules for internal record ID generation apply, unless an
+ optional <literal>recordIdNumber</literal> &zebra; internal ID or a
+ <literal>recordIdOpaque</literal> string identifier is assigned.
+ The default ID generation is
+ configured using the <literal>recordId:</literal> from
+ <filename>zebra.cfg</filename>.
+ See <xref linkend="zebra-cfg"/>.
+ </para>
+
+ <para>
+ Setting of the <literal>recordIdNumber</literal> parameter,
+ which must be an existing &zebra; internal system ID number, is not
+ allowed during any <literal>recordInsert</literal> or
+ <literal>specialUpdate</literal> action resulting in fresh record
+ inserts.
+ </para>
+
+ <para>
+ When retrieving existing
+ records indexed with &acro.grs1; indexing filters, the &zebra; internal
+ ID number is returned in the field
+ <literal>/*/id:idzebra/localnumber</literal> in the namespace
+ <literal>xmlns:id="http://www.indexdata.dk/zebra/"</literal>,
+ where it can be picked up for later record updates or deletes.
+ </para>
+
+ <para>
+ A new element set for retrieval of internal record
+ data has been added, which can be used to access minimal records
+ containing only the <literal>recordIdNumber</literal> &zebra;
+ internal ID, or the <literal>recordIdOpaque</literal> string
+ identifier. This works for any indexing filter used.
+ See <xref linkend="special-retrieval"/>.
+ </para>
+
+ <para>
+ The <literal>recordIdOpaque</literal> string parameter
+ is an client-supplied, opaque record
+ identifier, which may be used under
+ insert, update and delete operations. The
+ client software is responsible for assigning these to
+ records. This identifier will
+ replace zebra's own automagic identifier generation with a unique
+ mapping from <literal>recordIdOpaque</literal> to the
+ &zebra; internal <literal>recordIdNumber</literal>.
+ <emphasis>The opaque <literal>recordIdOpaque</literal> string
+ identifiers
+ are not visible in retrieval records, nor are
+ searchable, so the value of this parameter is
+ questionable. It serves mostly as a convenient mapping from
+ application domain string identifiers to &zebra; internal ID's.
+ </emphasis>
+ </para>
+ </sect2>
+
+
+ <sect2 id="administration-extended-services-yaz-client">
+ <title>Extended services from yaz-client</title>
+
<para>
We can now start a yaz-client admin session and create a database:
<screen>
]]>
</screen>
Now the <literal>Default</literal> database was created,
- we can insert an XML file (esdd0006.grs
+ we can insert an &acro.xml; file (esdd0006.grs
from example/gils/records) and index it:
<screen>
<![CDATA[
- Z> update insert 1 esdd0006.grs
+ Z> update insert id1234 esdd0006.grs
]]>
</screen>
- The 3rd parameter - <literal>1</literal> here -
- is the opaque record ID from <literal>Ext update</literal>.
- It a record ID that <emphasis>we</emphasis> assign to the record
- in question. If we do not
- assign one, the usual rules for match apply (recordId: from zebra.cfg).
+ The 3rd parameter - <literal>id1234</literal> here -
+ is the <literal>recordIdOpaque</literal> package field.
</para>
<para>
Actually, we should have a way to specify "no opaque record id" for
</screen>
</para>
<para>
- Let's delete the beast:
+ Let's delete the beast, using the same
+ <literal>recordIdOpaque</literal> string parameter:
<screen>
<![CDATA[
- Z> update delete 1
+ Z> update delete id1234
No last record (update ignored)
Z> update delete 1 esdd0006.grs
Got extended services response
after each update session in order write your changes from the
shadow to the life register space.
</para>
+ </sect2>
- </sect1>
+ <sect2 id="administration-extended-services-yaz-php">
+ <title>Extended services from yaz-php</title>
+
+ <para>
+ Extended services are also available from the &yaz; &acro.php; client layer. An
+ example of an &yaz;-&acro.php; extended service transaction is given here:
+ <screen>
+ <![CDATA[
+ $record = '<record><title>A fine specimen of a record</title></record>';
+
+ $options = array('action' => 'recordInsert',
+ 'syntax' => 'xml',
+ 'record' => $record,
+ 'databaseName' => 'mydatabase'
+ );
+ yaz_es($yaz, 'update', $options);
+ yaz_es($yaz, 'commit', array());
+ yaz_wait();
+
+ if ($error = yaz_error($yaz))
+ echo "$error";
+ ]]>
+ </screen>
+ </para>
+ </sect2>
- <sect1 id="gfs-config">
- <title>YAZ Frontend Virtual Hosts</title>
+ <sect2 id="administration-extended-services-debugging">
+ <title>Extended services debugging guide</title>
<para>
- <command>zebrasrv</command> uses the YAZ server frontend and does
- support multiple virtual servers behind multiple listening sockets.
+ When debugging ES over PHP we recommend the following order of tests:
</para>
- &zebrasrv-virtual;
-
- <para>
- Section "Virtual Hosts" in the YAZ manual.
- <filename>http://www.indexdata.dk/yaz/doc/server.vhosts.tkl</filename>
- </para>
- </sect1>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Make sure you have a nice record on your filesystem, which you can
+ index from the filesystem by use of the zebraidx command.
+ Do it exactly as you planned, using one of the GRS-1 filters,
+ or the DOMXML filter.
+ When this works, proceed.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Check that your server setup is OK before you even coded one single
+ line PHP using ES.
+ Take the same record form the file system, and send as ES via
+ <literal>yaz-client</literal> like described in
+ <xref linkend="administration-extended-services-yaz-client"/>,
+ and
+ remember the <literal>-a</literal> option which tells you what
+ goes over the wire! Notice also the section on permissions:
+ try
+ <screen>
+ perm.anonymous: rw
+ </screen>
+ in <literal>zebra.cfg</literal> to make sure you do not run into
+ permission problems (but never expose such an insecure setup on the
+ internet!!!). Then, make sure to set the general
+ <literal>recordType</literal> instruction, pointing correctly
+ to the GRS-1 filters,
+ or the DOMXML filters.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If you insist on using the <literal>sysno</literal> in the
+ <literal>recordIdNumber</literal> setting,
+ please make sure you do only updates and deletes. Zebra's internal
+ system number is not allowed for
+ <literal>recordInsert</literal> or
+ <literal>specialUpdate</literal> actions
+ which result in fresh record inserts.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If <literal>shadow register</literal> is enabled in your
+ <literal>zebra.cfg</literal>, you must remember running the
+ <screen>
+ Z> adm-commit
+ </screen>
+ command as well.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ If this works, then proceed to do the same thing in your PHP script.
+ </para>
+ </listitem>
+ </itemizedlist>
- <sect1 id="administration-cql-to-pqf">
- <title>Server Side CQL to PQF Query Translation</title>
- <para>
- Using the
- <literal><cql2rpn>l2rpn.txt</cql2rpn></literal>
- YAZ Frontend Virtual
- Hosts option, one can configure
- the YAZ Frontend CQL-to-PQF
- converter, specifying the interpretation of various
- <ulink url="http://www.loc.gov/standards/sru/cql/">CQL</ulink>
- indexes, relations, etc. in terms of Type-1 query attributes.
- <!-- The yaz-client config file -->
- </para>
- <para>
- For example, using server-side CQL-to-PQF conversion, one might
- query a zebra server like this:
- <screen>
- <![CDATA[
- yaz-client localhost:9999
- Z> querytype cql
- Z> find text=(plant and soil)
- ]]>
- </screen>
- and - if properly configured - even static relevance ranking can
- be performed using CQL query syntax:
- <screen>
- <![CDATA[
- Z> find text = /relevant (plant and soil)
- ]]>
- </screen>
- </para>
- <para>
- By the way, the same configuration can be used to
- search using client-side CQL-to-PQF conversion:
- (the only difference is <literal>querytype cql2rpn</literal>
- instead of
- <literal>querytype cql</literal>, and the call specifying a local
- conversion file)
- <screen>
- <![CDATA[
- yaz-client -q local/cql2pqf.txt localhost:9999
- Z> querytype cql2rpn
- Z> find text=(plant and soil)
- ]]>
- </screen>
- </para>
+ </sect2>
- <para>
- Exhaustive information can be found in the
- Section "Specification of CQL to RPN mappings" in the YAZ manual.
- <ulink url="http://www.indexdata.dk/yaz/doc/tools.tkl#tools.cql.map">
- http://www.indexdata.dk/yaz/doc/tools.tkl#tools.cql.map</ulink>,
- and shall therefore not be repeated here.
- </para>
- <!--
- <para>
- See
- <ulink url="http://www.loc.gov/z3950/agency/zing/cql/dc-indexes.html">
- http://www.loc.gov/z3950/agency/zing/cql/dc-indexes.html</ulink>
- for the Maintenance Agency's work-in-progress mapping of Dublin Core
- indexes to Attribute Architecture (util, XD and BIB-2)
- attributes.
- </para>
- -->
</sect1>
-
-
</chapter>
<!-- Keep this comment at the end of the file
projects / idzebra-moved-to-github.git / blobdiff