<chapter id="querymodel">
- <!-- $Id: querymodel.xml,v 1.20 2006-06-30 14:48:04 heikki Exp $ -->
<title>Query Model</title>
- <sect1 id="querymodel-overview">
+ <section id="querymodel-overview">
<title>Query Model Overview</title>
- <sect2 id="querymodel-query-languages">
+ <section id="querymodel-query-languages">
<title>Query Languages</title>
<para>
- Zebra is born as a networking Information Retrieval engine adhering
+ &zebra; is born as a networking Information Retrieval engine adhering
to the international standards
- <ulink url="&url.z39.50;">Z39.50</ulink> and
- <ulink url="&url.sru;">SRU</ulink>,
+ <ulink url="&url.z39.50;">&acro.z3950;</ulink> and
+ <ulink url="&url.sru;">&acro.sru;</ulink>,
and implement the
- <literal>type-1 Reverse Polish Notation (RPN)</literal> query
+ type-1 Reverse Polish Notation (&acro.rpn;) query
model defined there.
Unfortunately, this model has only defined a binary
encoded representation, which is used as transport packaging in
- the Z39.50 protocol layer. This representation is not human
+ the &acro.z3950; protocol layer. This representation is not human
readable, nor defines any convenient way to specify queries.
</para>
<para>
- Since the <literal>type-1 (RPN)</literal>
+ Since the type-1 (&acro.rpn;)
query structure has no direct, useful string
representation, every client application needs to provide some
form of mapping from a local query notation or representation to it.
</para>
- <sect3 id="querymodel-query-languages-pqf">
- <title>Prefix Query Format (PQF)</title>
+ <section id="querymodel-query-languages-pqf">
+ <title>Prefix Query Format (&acro.pqf;)</title>
<para>
Index Data has defined a textual representation in the
<ulink url="&url.yaz.pqf;">Prefix Query Format</ulink>, short
- <emphasis>PQF</emphasis>, which maps
+ <emphasis>&acro.pqf;</emphasis>, which maps
one-to-one to binary encoded
- <emphasis>type-1 RPN</emphasis> queries.
- PQF has been adopted by other
- parties developing Z39.50 software, and is often referred to as
- <literal>Prefix Query Notation</literal>, or in short
- <literal>PQN</literal>. See
- <xref linkend="querymodel-pqf"/> for further explanations and
- descriptions of Zebra's capabilities.
- </para>
- </sect3>
+ <emphasis>type-1 &acro.rpn;</emphasis> queries.
+ &acro.pqf; has been adopted by other
+ parties developing &acro.z3950; software, and is often referred to as
+ <emphasis>Prefix Query Notation</emphasis>, or in short
+ &acro.pqn;. See
+ <xref linkend="querymodel-rpn"/> for further explanations and
+ descriptions of &zebra;'s capabilities.
+ </para>
+ </section>
- <sect3 id="querymodel-query-languages-cql">
- <title>Common Query Language (CQL)</title>
+ <section id="querymodel-query-languages-cql">
+ <title>Common Query Language (&acro.cql;)</title>
<para>
- The query model of the type-1 RPN,
- expressed in PQF/PQN is natively supported.
- On the other hand, the default SRU
+ The query model of the type-1 &acro.rpn;,
+ expressed in &acro.pqf;/&acro.pqn; is natively supported.
+ On the other hand, the default &acro.sru;
web services <emphasis>Common Query Language</emphasis>
- <ulink url="&url.cql;">CQL</ulink> is not natively supported.
+ <ulink url="&url.cql;">&acro.cql;</ulink> is not natively supported.
</para>
<para>
- Zebra can be configured to understand and map CQL to PQF. See
+ &zebra; can be configured to understand and map &acro.cql; to &acro.pqf;. See
<xref linkend="querymodel-cql-to-pqf"/>.
</para>
- </sect3>
+ </section>
- </sect2>
+ </section>
- <sect2 id="querymodel-operation-types">
+ <section id="querymodel-operation-types">
<title>Operation types</title>
<para>
- Zebra supports all of the three different
- <literal>Z39.50/SRU</literal> operations defined in the
- standards: <literal>explain</literal>, <literal>search</literal>,
- and <literal>scan</literal>. A short description of the
+ &zebra; supports all of the three different
+ &acro.z3950;/&acro.sru; operations defined in the
+ standards: explain, search,
+ and scan. A short description of the
functionality and purpose of each is quite in order here.
</para>
- <sect3 id="querymodel-operation-type-explain">
+ <section id="querymodel-operation-type-explain">
<title>Explain Operation</title>
<para>
- The <emphasis>syntax</emphasis> of Z39.50/SRU queries is
+ The <emphasis>syntax</emphasis> of &acro.z3950;/&acro.sru; queries is
well known to any client, but the specific
<emphasis>semantics</emphasis> - taking into account a
particular servers functionalities and abilities - must be
discovered from case to case. Enters the
- <literal>explain</literal> operation, which provides the means
- for learning which
+ explain operation, which provides the means for learning which
<emphasis>fields</emphasis> (also called
<emphasis>indexes</emphasis> or <emphasis>access points</emphasis>)
are provided, which default parameter the server uses, which
of the general query model are supported.
</para>
<para>
- The Z39.50 embeds the <literal>explain</literal> operation
+ The &acro.z3950; embeds the explain operation
by performing a
- <literal>search</literal> in the magic
+ search in the magic
<literal>IR-Explain-1</literal> database;
see <xref linkend="querymodel-exp1"/>.
</para>
<para>
- In SRU, <literal>explain</literal> is an entirely separate
- operation, which returns an <literal>ZeeRex
- XML</literal> record according to the
+ In &acro.sru;, explain is an entirely separate
+ operation, which returns an ZeeRex &acro.xml; record according to the
structure defined by the protocol.
</para>
<para>
In both cases, the information gathered through
- <literal>explain</literal> operations can be used to
+ explain operations can be used to
auto-configure a client user interface to the servers
capabilities.
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-operation-type-search">
+ <section id="querymodel-operation-type-search">
<title>Search Operation</title>
<para>
Search and retrieve interactions are the raison d'être.
simple free text searches to nested complex boolean queries,
targeting specific indexes, and possibly enhanced with many
query semantic specifications. Search interactions are the heart
- and soul of Z39.50/SRU servers.
+ and soul of &acro.z3950;/&acro.sru; servers.
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-operation-type-scan">
+ <section id="querymodel-operation-type-scan">
<title>Scan Operation</title>
<para>
- The <literal>scan</literal> operation is a helper functionality,
+ The scan operation is a helper functionality,
which operates on one index or access point a time.
</para>
<para>
It provides
the means to investigate the content of specific indexes.
Scanning an index returns a handful of terms actually found in
- the indexes, and in addition the <literal>scan</literal>
+ the indexes, and in addition the scan
operation returns the number of documents indexed by each term.
A search client can use this information to propose proper
spelling of search terms, to auto-fill search boxes, or to
display controlled vocabularies.
</para>
- </sect3>
+ </section>
- </sect2>
+ </section>
- </sect1>
+ </section>
-
- <sect1 id="querymodel-pqf">
- <title>Prefix Query Format syntax and semantics</title>
+ <section id="querymodel-rpn">
+ <title>&acro.rpn; queries and semantics</title>
<para>
- The <ulink url="&url.yaz.pqf;">PQF grammar</ulink>
- is documented in the YAZ manual, and shall not be
- repeated here. This textual PQF representation
- is not transmistted to Zebra during search, but it is in the
- client mapped to the equivalent Z39.50 binary
+ The <ulink url="&url.yaz.pqf;">&acro.pqf; grammar</ulink>
+ is documented in the &yaz; manual, and shall not be
+ repeated here. This textual &acro.pqf; representation
+ is not transmitted to &zebra; during search, but it is in the
+ client mapped to the equivalent &acro.z3950; binary
query parse tree.
</para>
- <sect2 id="querymodel-pqf-tree">
- <title>PQF tree structure</title>
+ <section id="querymodel-rpn-tree">
+ <title>&acro.rpn; tree structure</title>
<para>
- The PQF parse tree - or the equivalent textual representation -
+ The &acro.rpn; parse tree - or the equivalent textual representation in &acro.pqf; -
may start with one specification of the
<emphasis>attribute set</emphasis> used. Following is a query
tree, which
- consists of <emphasis>atomic query parts (APT)</emphasis> or
+ consists of <emphasis>atomic query parts (&acro.apt;)</emphasis> or
<emphasis>named result sets</emphasis>, eventually
paired by <emphasis>boolean binary operators</emphasis>, and
finally <emphasis>recursively combined </emphasis> into
complex query trees.
</para>
- <sect3 id="querymodel-attribute-sets">
+ <section id="querymodel-attribute-sets">
<title>Attribute sets</title>
<para>
Attribute sets define the exact meaning and semantics of queries
- issued. Zebra comes with some predefined attribute set
+ issued. &zebra; comes with some predefined attribute set
definitions, others can easily be defined and added to the
configuration.
</para>
-
- <table id="querymodel-attribute-sets-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Attribute sets predefined in Zebra</caption>
-
+ <table id="querymodel-attribute-sets-table" frame="top">
+ <title>Attribute sets predefined in &zebra;</title>
+ <tgroup cols="4">
<thead>
- <tr>
- <td>Attribute set</td>
- <td>Short hand</td>
- <td>Status</td>
- <td>Notes</td>
- </tr>
- </thead>
-
+ <row>
+ <entry>Attribute set</entry>
+ <entry>&acro.pqf; notation (Short hand)</entry>
+ <entry>Status</entry>
+ <entry>Notes</entry>
+ </row>
+ </thead>
+
<tbody>
- <tr>
- <td><literal>Explain</literal></td>
- <td><literal>exp-1</literal></td>
- <td>Special attribute set used on the special automagic
+ <row>
+ <entry>Explain</entry>
+ <entry><literal>exp-1</literal></entry>
+ <entry>Special attribute set used on the special automagic
<literal>IR-Explain-1</literal> database to gain information on
server capabilities, database names, and database
- and semantics.</td>
- <td>predefined</td>
- </tr>
- <tr>
- <td><literal>Bib1</literal></td>
- <td><literal>bib-1</literal></td>
- <td>Standard PQF query language attribute set which defines the
- semantics of Z39.50 searching. In addition, all of the
- non-use attributes (type 2-9) define the hard-wired
- Zebra internal query
- processing.</td>
- <td>default</td>
- </tr>
- <tr>
- <td><literal>GILS</literal></td>
- <td><literal>gils</literal></td>
- <td>Extension to the <literal>Bib1</literal> attribute set.</td>
- <td>predefined</td>
- </tr>
- <!--
- <tr>
- <td><literal>IDXPATH</literal></td>
- <td><literal>idxpath</literal></td>
- <td>Hardwired XPATH like attribute set, only available for
- indexing with the GRS record model</td>
- <td>depreciated</td>
- </tr>
- -->
+ and semantics.</entry>
+ <entry>predefined</entry>
+ </row>
+ <row>
+ <entry>&acro.bib1;</entry>
+ <entry><literal>bib-1</literal></entry>
+ <entry>Standard &acro.pqf; query language attribute set which defines the
+ semantics of &acro.z3950; searching. In addition, all of the
+ non-use attributes (types 2-14) define the hard-wired
+ &zebra; internal query
+ processing.</entry>
+ <entry>default</entry>
+ </row>
+ <row>
+ <entry>GILS</entry>
+ <entry><literal>gils</literal></entry>
+ <entry>Extension to the &acro.bib1; attribute set.</entry>
+ <entry>predefined</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
- </sect3>
-
- <para>
- The <literal>use attributes (type 1)</literal> mappings the
- predefined attribute sets are found in the
- attribute set configuration files <filename>tab/*.att</filename>.
- </para>
-
- <note>
- The Zebra internal query processing is modeled after
- the <literal>Bib1</literal> attribute set, and the non-use
- attributes type 2-6 are hard-wired in. It is therefore essential
- to be familiar with <xref linkend="querymodel-bib1-nonuse"/>.
- </note>
-
+
+ <para>
+ The use attributes (type 1) mappings the
+ predefined attribute sets are found in the
+ attribute set configuration files <filename>tab/*.att</filename>.
+ </para>
+
+ <note>
+ <para>
+ The &zebra; internal query processing is modeled after
+ the &acro.bib1; attribute set, and the non-use
+ attributes type 2-6 are hard-wired in. It is therefore essential
+ to be familiar with <xref linkend="querymodel-bib1-nonuse"/>.
+ </para>
+ </note>
+
+ </section>
- <sect3 id="querymodel-boolean-operators">
+ <section id="querymodel-boolean-operators">
<title>Boolean operators</title>
<para>
A pair of sub query trees, or of atomic queries, is combined
Thus, boolean operators are always internal nodes in the query tree.
</para>
- <table id="querymodel-boolean-operators-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Boolean operators</caption>
+ <table id="querymodel-boolean-operators-table" frame="top">
+ <title>Boolean operators</title>
+ <tgroup cols="3">
<thead>
- <tr>
- <td>Keyword</td>
- <td>Operator</td>
- <td>Description</td>
- </tr>
- </thead>
+ <row>
+ <entry>Keyword</entry>
+ <entry>Operator</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
<tbody>
- <tr><td><literal>@and</literal></td>
- <td>binary <literal>AND</literal> operator</td>
- <td>Set intersection of two atomic queries hit sets</td>
- </tr>
- <tr><td><literal>@or</literal></td>
- <td>binary <literal>OR</literal> operator</td>
- <td>Set union of two atomic queries hit sets</td>
- </tr>
- <tr><td><literal>@not</literal></td>
- <td>binary <literal>AND NOT</literal> operator</td>
- <td>Set complement of two atomic queries hit sets</td>
- </tr>
- <tr><td><literal>@prox</literal></td>
- <td>binary <literal>PROXIMITY</literal> operator</td>
- <td>Set intersection of two atomic queries hit sets. In
- addition, the intersection set is purged for all
- documents which do not satisfy the requested query
- term proximity. Usually a proper subset of the AND
- operation.</td>
- </tr>
+ <row><entry><literal>@and</literal></entry>
+ <entry>binary AND operator</entry>
+ <entry>Set intersection of two atomic queries hit sets</entry>
+ </row>
+ <row><entry><literal>@or</literal></entry>
+ <entry>binary OR operator</entry>
+ <entry>Set union of two atomic queries hit sets</entry>
+ </row>
+ <row><entry><literal>@not</literal></entry>
+ <entry>binary AND NOT operator</entry>
+ <entry>Set complement of two atomic queries hit sets</entry>
+ </row>
+ <row><entry><literal>@prox</literal></entry>
+ <entry>binary PROXIMITY operator</entry>
+ <entry>Set intersection of two atomic queries hit sets. In
+ addition, the intersection set is purged for all
+ documents which do not satisfy the requested query
+ term proximity. Usually a proper subset of the AND
+ operation.</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
<para>
<emphasis>retrieval</emphasis>, taking proximity into account:
The hit set is a subset of the corresponding
AND query
- (see the <ulink url="&url.yaz.pqf;">PQF grammar</ulink> for
+ (see the <ulink url="&url.yaz.pqf;">&acro.pqf; grammar</ulink> for
details on the proximity operator):
<screen>
Z> find @prox 0 3 0 2 k 2 information retrieval
Z> find "information retrieval"
</screen>
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-atomic-queries">
- <title>Atomic queries (APT)</title>
+ <section id="querymodel-atomic-queries">
+ <title>Atomic queries (&acro.apt;)</title>
<para>
Atomic queries are the query parts which work on one access point
- only. These consist of <literal>an attribute list</literal>
- followed by a <literal>single term</literal> or a
- <literal>quoted term list</literal>, and are often called
- <emphasis>Attributes-Plus-Terms (APT)</emphasis> queries.
+ only. These consist of <emphasis>an attribute list</emphasis>
+ followed by a <emphasis>single term</emphasis> or a
+ <emphasis>quoted term list</emphasis>, and are often called
+ <emphasis>Attributes-Plus-Terms (&acro.apt;)</emphasis> queries.
</para>
<para>
- Atomic (APT) queries are always leaf nodes in the PQF query tree.
- UN-supplied non-use attributes type 2-9 are either inherited from
- higher nodes in the query tree, or are set to Zebra's default values.
+ Atomic (&acro.apt;) queries are always leaf nodes in the &acro.pqf; query tree.
+ UN-supplied non-use attributes types 2-12 are either inherited from
+ higher nodes in the query tree, or are set to &zebra;'s default values.
See <xref linkend="querymodel-bib1"/> for details.
</para>
- <table id="querymodel-atomic-queries-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Atomic queries (APT)</caption>
+ <table id="querymodel-atomic-queries-table" frame="top">
+ <title>Atomic queries (&acro.apt;)</title>
+ <tgroup cols="3">
<thead>
- <tr>
- <td>Name</td>
- <td>Type</td>
- <td>Notes</td>
- </tr>
+ <row>
+ <entry>Name</entry>
+ <entry>Type</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td><emphasis>attribute list</emphasis></td>
- <td>List of <literal>orthogonal</literal> attributes</td>
- <td>Any of the orthogonal attribute types may be omitted,
+ <row>
+ <entry><emphasis>attribute list</emphasis></entry>
+ <entry>List of <emphasis>orthogonal</emphasis> attributes</entry>
+ <entry>Any of the orthogonal attribute types may be omitted,
these are inherited from higher query tree nodes, or if not
- inherited, are set to the default Zebra configuration values.
- </td>
- </tr>
- <tr>
- <td><emphasis>term</emphasis></td>
- <td>single <literal>term</literal>
- or <literal>quoted term list</literal> </td>
- <td>Here the search terms or list of search terms is added
- to the query</td>
- </tr>
+ inherited, are set to the default &zebra; configuration values.
+ </entry>
+ </row>
+ <row>
+ <entry><emphasis>term</emphasis></entry>
+ <entry>single <emphasis>term</emphasis>
+ or <emphasis>quoted term list</emphasis> </entry>
+ <entry>Here the search terms or list of search terms is added
+ to the query</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
<para>
Querying for the term <emphasis>information</emphasis> in the
</para>
<para>
- The <literal>scan</literal> operation is only supported with
- atomic APT queries, as it is bound to one access point at a
+ The <emphasis>scan</emphasis> operation is only supported with
+ atomic &acro.apt; queries, as it is bound to one access point at a
time. Boolean query trees are not allowed during
- <literal>scan</literal>.
+ <emphasis>scan</emphasis>.
</para>
<para>
Z> scan @attr 1=4 debussy
</screen>
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-resultset">
+ <section id="querymodel-resultset">
<title>Named Result Sets</title>
<para>
- Named result sets are supported in Zebra, and result sets can be
+ Named result sets are supported in &zebra;, and result sets can be
used as operands without limitations. It follows that named
- result sets are leaf nodes in the PQF query tree, exactly as
- atomic APT queries are.
+ result sets are leaf nodes in the &acro.pqf; query tree, exactly as
+ atomic &acro.apt; queries are.
</para>
<para>
After the execution of a search, the result set is available at
<para>
Defining a named result set and re-using it in the next query,
- using <literal>yaz-client</literal>. Notice that the client, not
- the server, assigns the string <literal>'1'</literal> to the
+ using <application>yaz-client</application>. Notice that the client, not
+ the server, assigns the string '1' to the
named result set.
<screen>
Z> f @attr 1=4 mozart
</para>
<note>
- Named result sets are only supported by the Z39.50 protocol.
- The SRU web service is stateless, and therefore the notion of
- named result sets does not exist when accessing a Zebra server by
- the SRU protocol.
+ <para>
+ Named result sets are only supported by the &acro.z3950; protocol.
+ The &acro.sru; web service is stateless, and therefore the notion of
+ named result sets does not exist when accessing a &zebra; server by
+ the &acro.sru; protocol.
+ </para>
</note>
- </sect3>
-
-
- <sect3 id="querymodel-use-string">
- <title>Zebra's special access point of type 'string'</title>
+ </section>
+
+ <section id="querymodel-use-string">
+ <title>&zebra;'s special access point of type 'string'</title>
<para>
- The numeric <literal>use (type 1)</literal> attribute is usually
+ The numeric <emphasis>use (type 1)</emphasis> attribute is usually
referred to from a given
- attribute set. In addition, Zebra let you use
+ attribute set. In addition, &zebra; let you use
<emphasis>any internal index
name defined in your configuration</emphasis>
as use attribute value. This is a great feature for
debugging, and when you do
not need the complexity of defined use attribute values. It is
- the preferred way of accessing Zebra indexes directly.
+ the preferred way of accessing &zebra; indexes directly.
</para>
<para>
Finding all documents which have the term list "information
- retrieval" in an Zebra index, using it's internal full string
+ retrieval" in an &zebra; index, using its internal full string
name. Scanning the same index.
<screen>
Z> find @attr 1=sometext "information retrieval"
</para>
<para>
Searching or scanning
- the bib-1 use attribute 54 using it's string name:
+ the bib-1 use attribute 54 using its string name:
<screen>
Z> find @attr 1=Code-language eng
Z> scan @attr 1=Code-language ""
<para>
It is possible to search
in any silly string index - if it's defined in your
- indexation rules and can be parsed by the PQF parser.
+ indexing rules and can be parsed by the &acro.pqf; parser.
This is definitely not the recommended use of
this facility, as it might confuse your users with some very
unexpected results.
</para>
<para>
See also <xref linkend="querymodel-pqf-apt-mapping"/> for details, and
- <xref linkend="server-sru"/>
- for the SRU PQF query extension using string names as a fast
+ <xref linkend="zebrasrv-sru"/>
+ for the &acro.sru; &acro.pqf; query extension using string names as a fast
debugging facility.
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-use-xpath">
- <title>Zebra's special access point of type 'XPath'
- for GRS filters</title>
+ <section id="querymodel-use-xpath">
+ <title>&zebra;'s special access point of type 'XPath'
+ for &acro.grs1; filters</title>
<para>
As we have seen above, it is possible (albeit seldom a great
idea) to emulate
<ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink> based
- search by defining <literal>use (type 1)</literal>
+ search by defining <emphasis>use (type 1)</emphasis>
<emphasis>string</emphasis> attributes which in appearance
<emphasis>resemble XPath queries</emphasis>. There are two
problems with this approach: first, the XPath-look-alike has to
- be defined at indexation time, no new undefined
+ be defined at indexing time, no new undefined
XPath queries can entered at search time, and second, it might
confuse users very much that an XPath-alike index name in fact
- gets populated from a possible entirely different XML element
+ gets populated from a possible entirely different &acro.xml; element
than it pretends to access.
</para>
<para>
- When using the <literal>GRS Record Model</literal>
- (see <xref linkend="record-model-grs"/>), we have the
+ When using the &acro.grs1; Record Model
+ (see <xref linkend="grs"/>), we have the
possibility to embed <emphasis>life</emphasis>
XPath expressions
- in the PQF queries, which are here called
- <literal>use (type 1)</literal> <emphasis>xpath</emphasis>
+ in the &acro.pqf; queries, which are here called
+ <emphasis>use (type 1)</emphasis> <emphasis>xpath</emphasis>
attributes. You must enable the
<literal>xpath enable</literal> directive in your
<literal>.abs</literal> configuration files.
</para>
<note>
- Only a <emphasis>very</emphasis> restricted subset of the
- <ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink>
- standard is supported as the GRS record model is simpler than
- a full XML DOM structure. See the following examples for
- possibilities.
+ <para>
+ Only a <emphasis>very</emphasis> restricted subset of the
+ <ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink>
+ standard is supported as the &acro.grs1; record model is simpler than
+ a full &acro.xml; &acro.dom; structure. See the following examples for
+ possibilities.
+ </para>
</note>
<para>
Finding all documents which have the term "content"
- inside a text node found in a specific XML DOM
+ inside a text node found in a specific &acro.xml; &acro.dom;
<emphasis>subtree</emphasis>, whose starting element is
addressed by XPath.
<screen>
<para>
Filter the addressing XPath by a predicate working on exact
string values in
- attributes (in the XML sense) can be done: return all those docs which
+ attributes (in the &acro.xml; sense) can be done: return all those docs which
have the term "english" contained in one of all text sub nodes of
the subtree defined by the XPath
<literal>/record/title[@lang='en']</literal>. And similar
</screen>
</para>
<para>
- Escaping PQF keywords and other non-parseable XPath constructs
- with <literal>'{ }'</literal> to prevent syntax errors:
+ Escaping &acro.pqf; keywords and other non-parseable XPath constructs
+ with <literal>'{ }'</literal> to prevent client-side &acro.pqf; parsing
+ syntax errors:
<screen>
Z> find @attr {1=/root/first[@attr='danish']} content
Z> find @attr {1=/record/@set} oai
</screen>
</para>
<warning>
- It is worth mentioning that these dynamic performed XPath
- queries are a performance bottleneck, as no optimized
- specialized indexes can be used. Therefore, avoid the use of
- this facility when speed is essential, and the database content
- size is medium to large.
+ <para>
+ It is worth mentioning that these dynamic performed XPath
+ queries are a performance bottleneck, as no optimized
+ specialized indexes can be used. Therefore, avoid the use of
+ this facility when speed is essential, and the database content
+ size is medium to large.
+ </para>
</warning>
-
- </sect3>
-
- </sect2>
+ </section>
+ </section>
- <sect2 id="querymodel-exp1">
+ <section id="querymodel-exp1">
<title>Explain Attribute Set</title>
<para>
- The Z39.50 standard defines the
+ The &acro.z3950; standard defines the
<ulink url="&url.z39.50.explain;">Explain</ulink> attribute set
- <literal>Exp-1</literal>, which is used to discover information
+ Exp-1, which is used to discover information
about a server's search semantics and functional capabilities
- Zebra exposes a "classic"
+ &zebra; exposes a "classic"
Explain database by base name <literal>IR-Explain-1</literal>, which
is populated with system internal information.
</para>
<para>
The attribute-set <literal>exp-1</literal> consists of a single
- <literal>use attribute (type 1)</literal>.
+ use attribute (type 1).
</para>
<para>
In addition, the non-Use
- <literal>bib-1</literal> attributes, that is, the types
- <literal>Relation</literal>, <literal>Position</literal>,
- <literal>Structure</literal>, <literal>Truncation</literal>,
- and <literal>Completeness</literal> are imported from
- the <literal>bib-1</literal> attribute set, and may be used
+ &acro.bib1; attributes, that is, the types
+ <emphasis>Relation</emphasis>, <emphasis>Position</emphasis>,
+ <emphasis>Structure</emphasis>, <emphasis>Truncation</emphasis>,
+ and <emphasis>Completeness</emphasis> are imported from
+ the &acro.bib1; attribute set, and may be used
within any explain query.
</para>
- <sect3 id="querymodel-exp1-use">
+ <section id="querymodel-exp1-use">
<title>Use Attributes (type = 1)</title>
<para>
The following Explain search attributes are supported:
</para>
<para>
See <filename>tab/explain.att</filename> and the
- <ulink url="&url.z39.50;">Z39.50</ulink> standard
+ <ulink url="&url.z39.50;">&acro.z3950;</ulink> standard
for more information.
</para>
- </sect3>
+ </section>
- <sect3>
+ <section id="querymodel-examples">
<title>Explain searches with yaz-client</title>
<para>
Classic Explain only defines retrieval of Explain information
- via ASN.1. Practically no Z39.50 clients supports this. Fortunately
- they don't have to - Zebra allows retrieval of this information
+ via ASN.1. Practically no &acro.z3950; clients supports this. Fortunately
+ they don't have to - &zebra; allows retrieval of this information
in other formats:
- <literal>SUTRS</literal>, <literal>XML</literal>,
- <literal>GRS-1</literal> and <literal>ASN.1</literal> Explain.
+ &acro.sutrs;, &acro.xml;,
+ &acro.grs1; and <literal>ASN.1</literal> Explain.
</para>
<para>
<para>
Get attribute details record for database
<literal>Default</literal>.
- This query is very useful to study the internal Zebra indexes.
+ This query is very useful to study the internal &zebra; indexes.
If records have been indexed using the <literal>alvis</literal>
- XSLT filter, the string representation names of the known indexes can be
+ &acro.xslt; filter, the string representation names of the known indexes can be
found.
<screen>
Z> base IR-Explain-1
Z> find @attrset exp1 @and @attr 1=1 attributedetails @attr 1=3 Default
</screen>
</para>
- </sect3>
+ </section>
- </sect2>
+ </section>
- <sect2 id="querymodel-bib1">
- <title>Bib1 Attribute Set</title>
+ <section id="querymodel-bib1">
+ <title>&acro.bib1; Attribute Set</title>
<para>
Most of the information contained in this section is an excerpt of
- the <literal>ATTRIBUTE SET BIB-1 (Z39.50-1995)
- SEMANTICS</literal>,
- found at <ulink url="&url.z39.50.attset.bib1.1995;">. The BIB-1
+ the ATTRIBUTE SET &acro.bib1; (&acro.z3950;-1995) SEMANTICS
+ found at <ulink url="&url.z39.50.attset.bib1.1995;">. The &acro.bib1;
Attribute Set Semantics</ulink> from 1995, also in an updated
- <ulink url="&url.z39.50.attset.bib1;">Bib-1
+ <ulink url="&url.z39.50.attset.bib1;">&acro.bib1;
Attribute Set</ulink>
version from 2003. Index Data is not the copyright holder of this
information, except for the configuration details, the listing of
- Zebra's capabilities, and the example queries.
+ &zebra;'s capabilities, and the example queries.
</para>
- <sect3 id="querymodel-bib1-use">
+ <section id="querymodel-bib1-use">
<title>Use Attributes (type 1)</title>
<para>
<filename>tab/dan1.att</filename>,
<filename>tab/explain.att</filename>, and
<filename>tab/gils.att</filename>.
+ </para>
+ <para>
+ For example, some few &acro.bib1; use
+ attributes from the <filename>tab/bib1.att</filename> are:
+ <screen>
+ att 1 Personal-name
+ att 2 Corporate-name
+ att 3 Conference-name
+ att 4 Title
+ ...
+ att 1009 Subject-name-personal
+ att 1010 Body-of-text
+ att 1011 Date/time-added-to-db
+ ...
+ att 1016 Any
+ att 1017 Server-choice
+ att 1018 Publisher
+ ...
+ att 1035 Anywhere
+ att 1036 Author-Title-Subject
+ </screen>
+ </para>
+ <para>
New attribute sets can be added by adding new
<filename>tab/*.att</filename> configuration files, which need to
- be sourced in the main configuration <filename>zebra.cfg</filename>.
+ be sourced in the main configuration <filename>zebra.cfg</filename>.
</para>
-
<para>
- In addition, Zebra allows the access of
+ In addition, &zebra; allows the access of
<emphasis>internal index names</emphasis> and <emphasis>dynamic
XPath</emphasis> as use attributes; see
<xref linkend="querymodel-use-string"/> and
Z> scan @attr 1=4 information
</screen>
</para>
- </sect3>
+ </section>
- </sect2>
+ </section>
- <sect2 id="querymodel-bib1-nonuse">
- <title>Zebra general Bib1 Non-Use Attributes (type 2-6)</title>
-
- <sect3 id="querymodel-bib1-relation">
+ <section id="querymodel-bib1-nonuse">
+ <title>&zebra; general Bib1 Non-Use Attributes (type 2-6)</title>
+
+ <section id="querymodel-bib1-relation">
<title>Relation Attributes (type 2)</title>
<para>
side of the relation), e.g., Date-publication <= 1975.
</para>
- <table id="querymodel-bib1-relation-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Relation Attributes (type 2)</caption>
- <thead>
- <tr>
- <td>Relation</td>
- <td>Value</td>
- <td>Notes</td>
- </tr>
+ <table id="querymodel-bib1-relation-table" frame="top">
+ <title>Relation Attributes (type 2)</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Relation</entry>
+ <entry>Value</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td> Less than</td>
- <td>1</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Less than or equal</td>
- <td>2</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Equal</td>
- <td>3</td>
- <td>default</td>
- </tr>
- <tr>
- <td>Greater or equal</td>
- <td>4</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Greater than</td>
- <td>5</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Not equal</td>
- <td>6</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Phonetic</td>
- <td>100</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Stem</td>
- <td>101</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Relevance</td>
- <td>102</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>AlwaysMatches</td>
- <td>103</td>
- <td>supported</td>
- </tr>
+ <row>
+ <entry>Less than</entry>
+ <entry>1</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Less than or equal</entry>
+ <entry>2</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Equal</entry>
+ <entry>3</entry>
+ <entry>default</entry>
+ </row>
+ <row>
+ <entry>Greater or equal</entry>
+ <entry>4</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Greater than</entry>
+ <entry>5</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Not equal</entry>
+ <entry>6</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Phonetic</entry>
+ <entry>100</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Stem</entry>
+ <entry>101</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Relevance</entry>
+ <entry>102</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>AlwaysMatches</entry>
+ <entry>103</entry>
+ <entry>supported *</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
-
+ <note>
+ <para>
+ AlwaysMatches searches are only supported if alwaysmatches indexing
+ has been enabled. See <xref linkend="default-idx-file"/>
+ </para>
+ </note>
+
<para>
- The relation attributes
- <literal>1-5</literal> are supported and work exactly as
+ The relation attributes 1-5 are supported and work exactly as
expected.
All ordering operations are based on a lexicographical ordering,
- <emphasis>expect</emphasis> when the
- <literal>structure attribute numeric (109)</literal> is used. In
+ <emphasis>except</emphasis> when the
+ structure attribute numeric (109) is used. In
this case, ordering is numerical. See
<xref linkend="querymodel-bib1-structure"/>.
<screen>
- Z> find @attr 1=Title @attr 2=1 music
+ Z> find @attr 1=Title @attr 2=1 music
...
Number of hits: 11745, setno 1
...
- Z> find @attr 1=Title @attr 2=2 music
+ Z> find @attr 1=Title @attr 2=2 music
...
Number of hits: 11771, setno 2
...
- Z> find @attr 1=Title @attr 2=3 music
+ Z> find @attr 1=Title @attr 2=3 music
...
Number of hits: 532, setno 3
...
- Z> find @attr 1=Title @attr 2=4 music
+ Z> find @attr 1=Title @attr 2=4 music
...
Number of hits: 11463, setno 4
...
- Z> find @attr 1=Title @attr 2=5 music
+ Z> find @attr 1=Title @attr 2=5 music
...
Number of hits: 11419, setno 5
</screen>
<para>
The relation attribute
- <literal>Relevance (102)</literal> is supported, see
+ <emphasis>Relevance (102)</emphasis> is supported, see
<xref linkend="administration-ranking"/> for full information.
</para>
<para>
The relation attribute
- <literal>AlwaysMatches (103)</literal> is in the default
+ <emphasis>AlwaysMatches (103)</emphasis> is in the default
configuration
supported in conjecture with structure attribute
- <literal>Phrase (1)</literal> (which may be omitted by
+ <emphasis>Phrase (1)</emphasis> (which may be omitted by
default).
It can be configured to work with other structure attributes,
see the configuration file
<xref linkend="querymodel-pqf-apt-mapping"/>.
</para>
<para>
- <literal>AlwaysMatches (103)</literal> is a
+ <emphasis>AlwaysMatches (103)</emphasis> is a
great way to discover how many documents have been indexed in a
given field. The search term is ignored, but needed for correct
- PQF syntax. An empty search term may be supplied.
+ &acro.pqf; syntax. An empty search term may be supplied.
<screen>
Z> find @attr 1=Title @attr 2=103 ""
Z> find @attr 1=Title @attr 2=103 @attr 4=1 ""
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-bib1-position">
+ <section id="querymodel-bib1-position">
<title>Position Attributes (type 3)</title>
<para>
within the field or subfield in which it appears.
</para>
- <table id="querymodel-bib1-position-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Position Attributes (type 3)</caption>
- <thead>
- <tr>
- <td>Position</td>
- <td>Value</td>
- <td>Notes</td>
- </tr>
+ <table id="querymodel-bib1-position-table" frame="top">
+ <title>Position Attributes (type 3)</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Position</entry>
+ <entry>Value</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td>First in field </td>
- <td>1</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>First in subfield</td>
- <td>2</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Any position in field</td>
- <td>3</td>
- <td>default</td>
- </tr>
+ <row>
+ <entry>First in field </entry>
+ <entry>1</entry>
+ <entry>supported *</entry>
+ </row>
+ <row>
+ <entry>First in subfield</entry>
+ <entry>2</entry>
+ <entry>supported *</entry>
+ </row>
+ <row>
+ <entry>Any position in field</entry>
+ <entry>3</entry>
+ <entry>default</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
-
- <para>
- The position attribute values <literal>first in field (1)</literal>,
- and <literal>first in subfield(2)</literal> are unsupported.
- Using them does not trigger an error, but silently defaults to
- <literal>any position in field (3)</literal>.
- <!-- It should -->
+
+ <note>
+ <para>
+ &zebra; only supports first-in-field seaches if the
+ <literal>firstinfield</literal> is enabled for the index
+ Refer to <xref linkend="default-idx-file"/>.
+ &zebra; does not distinguish between first in field and
+ first in subfield. They result in the same hit count.
+ Searching for first position in (sub)field in only supported in &zebra;
+ 2.0.2 and later.
</para>
- </sect3>
-
- <sect3 id="querymodel-bib1-structure">
+ </note>
+ </section>
+
+ <section id="querymodel-bib1-structure">
<title>Structure Attributes (type 4)</title>
<para>
The structure attribute specifies the type of search
term. This causes the search to be mapped on
- different Zebra internal indexes, which must have been defined
+ different &zebra; internal indexes, which must have been defined
at index time.
</para>
<para>
The possible values of the
<literal>structure attribute (type 4)</literal> can be defined
- using the configuration file <filename>
- tab/default.idx</filename>.
+ using the configuration file <filename>tab/default.idx</filename>.
The default configuration is summarized in this table.
</para>
- <table id="querymodel-bib1-structure-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Structure Attributes (type 4)</caption>
- <thead>
- <tr>
- <td>Structure</td>
- <td>Value</td>
- <td>Notes</td>
- </tr>
+ <table id="querymodel-bib1-structure-table" frame="top">
+ <title>Structure Attributes (type 4)</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Structure</entry>
+ <entry>Value</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td>Phrase </td>
- <td>1</td>
- <td>default</td>
- </tr>
- <tr>
- <td>Word</td>
- <td>2</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Key</td>
- <td>3</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Year</td>
- <td>4</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Date (normalized)</td>
- <td>5</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Word list</td>
- <td>6</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Date (un-normalized)</td>
- <td>100</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Name (normalized) </td>
- <td>101</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Name (un-normalized) </td>
- <td>102</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Structure</td>
- <td>103</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Urx</td>
- <td>104</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Free-form-text</td>
- <td>105</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Document-text</td>
- <td>106</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Local-number</td>
- <td>107</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>String</td>
- <td>108</td>
- <td>unsupported</td>
- </tr>
- <tr>
- <td>Numeric string</td>
- <td>109</td>
- <td>supported</td>
- </tr>
+ <row>
+ <entry>Phrase </entry>
+ <entry>1</entry>
+ <entry>default</entry>
+ </row>
+ <row>
+ <entry>Word</entry>
+ <entry>2</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Key</entry>
+ <entry>3</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Year</entry>
+ <entry>4</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Date (normalized)</entry>
+ <entry>5</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Word list</entry>
+ <entry>6</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Date (un-normalized)</entry>
+ <entry>100</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Name (normalized) </entry>
+ <entry>101</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Name (un-normalized) </entry>
+ <entry>102</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Structure</entry>
+ <entry>103</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Urx</entry>
+ <entry>104</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Free-form-text</entry>
+ <entry>105</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Document-text</entry>
+ <entry>106</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Local-number</entry>
+ <entry>107</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>String</entry>
+ <entry>108</entry>
+ <entry>unsupported</entry>
+ </row>
+ <row>
+ <entry>Numeric string</entry>
+ <entry>109</entry>
+ <entry>supported</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
-
-
<para>
The structure attribute values
<literal>Word list (6)</literal>
is supported, and maps to the boolean <literal>AND</literal>
combination of words supplied. The word list is useful when
- google-like bag-of-word queries need to be translated from a GUI
- query language to PQF. For example, the following queries
+ Google-like bag-of-word queries need to be translated from a GUI
+ query language to &acro.pqf;. For example, the following queries
are equivalent:
<screen>
Z> find @attr 1=Title @attr 4=6 "mozart amadeus"
Z> find @attr 1=Body-of-text @attr 2=102 @attr 4=105 "bach salieri teleman"
</screen>
</para>
-
+
<para>
The structure attribute value
<literal>Local number (107)</literal>
- is supported, and maps always to the Zebra internal document ID,
+ is supported, and maps always to the &zebra; internal document ID,
irrespectively which use attribute is specified. The following queries
have exactly the same unique record in the hit set:
<screen>
<screen>
Z> find @attr 4=109 @attr 2=5 @attr gils 1=2038 -114
</screen>
- </para>
+ </para>
<note>
- The exact mapping between PQF queries and Zebra internal indexes
- and index types is explained in
+ <para>
+ The exact mapping between &acro.pqf; queries and &zebra; internal indexes
+ and index types is explained in
<xref linkend="querymodel-pqf-apt-mapping"/>.
- </note>
-
- </sect3>
+ </para>
+ </note>
+ </section>
+
- <sect3 id="querymodel-bib1-truncation">
+ <section id="querymodel-bib1-truncation">
<title>Truncation Attributes (type = 5)</title>
<para>
document hit set of a search query.
</para>
- <table id="querymodel-bib1-truncation-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Truncation Attributes (type 5)</caption>
- <thead>
- <tr>
- <td>Truncation</td>
- <td>Value</td>
- <td>Notes</td>
- </tr>
+ <table id="querymodel-bib1-truncation-table" frame="top">
+ <title>Truncation Attributes (type 5)</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Truncation</entry>
+ <entry>Value</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td>Right truncation </td>
- <td>1</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Left truncation</td>
- <td>2</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Left and right truncation</td>
- <td>3</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>Do not truncate</td>
- <td>100</td>
- <td>default</td>
- </tr>
- <tr>
- <td>Process # in search term</td>
- <td>101</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>RegExpr-1 </td>
- <td>102</td>
- <td>supported</td>
- </tr>
- <tr>
- <td>RegExpr-2</td>
- <td>103</td>
- <td>supported</td>
- </tr>
+ <row>
+ <entry>Right truncation </entry>
+ <entry>1</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Left truncation</entry>
+ <entry>2</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Left and right truncation</entry>
+ <entry>3</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>Do not truncate</entry>
+ <entry>100</entry>
+ <entry>default</entry>
+ </row>
+ <row>
+ <entry>Process # in search term</entry>
+ <entry>101</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>RegExpr-1 </entry>
+ <entry>102</entry>
+ <entry>supported</entry>
+ </row>
+ <row>
+ <entry>RegExpr-2</entry>
+ <entry>103</entry>
+ <entry>supported</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
<para>
<para>
The truncation attribute value
- <literal>Regexp-2 (103) </literal> is a Zebra specific extension
+ <literal>Regexp-2 (103) </literal> is a &zebra; specific extension
which allows <emphasis>fuzzy</emphasis> matches. One single
error in spelling of search terms is allowed, i.e., a document
is hit if it includes a term which can be mapped to the used
...
</screen>
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-bib1-completeness">
+ <section id="querymodel-bib1-completeness">
<title>Completeness Attributes (type = 6)</title>
(<literal>Complete field (3)</literal>).
</para>
- <table id="querymodel-bib1-completeness-table"
- frame="all" rowsep="1" colsep="1" align="center">
- <caption>Completeness Attributes (type = 6)</caption>
- <thead>
- <tr>
- <td>Completeness</td>
- <td>Value</td>
- <td>Notes</td>
- </tr>
+ <table id="querymodel-bib1-completeness-table" frame="top">
+ <title>Completeness Attributes (type = 6)</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Completeness</entry>
+ <entry>Value</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td>Incomplete subfield</td>
- <td>1</td>
- <td>default</td>
- </tr>
- <tr>
- <td>Complete subfield</td>
- <td>2</td>
- <td>depreciated</td>
- </tr>
- <tr>
- <td>Complete field</td>
- <td>3</td>
- <td>supported</td>
- </tr>
+ <row>
+ <entry>Incomplete subfield</entry>
+ <entry>1</entry>
+ <entry>default</entry>
+ </row>
+ <row>
+ <entry>Complete subfield</entry>
+ <entry>2</entry>
+ <entry>deprecated</entry>
+ </row>
+ <row>
+ <entry>Complete field</entry>
+ <entry>3</entry>
+ <entry>supported</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
<para>
</para>
<para>
<literal>Incomplete subfield (1)</literal> is the default, and
- makes Zebra use
+ makes &zebra; use
register <literal>type="w"</literal>, whereas
<literal>Complete field (3)</literal> triggers
search and scan in index <literal>type="p"</literal>.
</para>
<para>
- The <literal>Complete subfield (2)</literal> is a reminiscens
- from the happy <literal>MARC</literal>
- binary format days. Zebra does not support it, but maps silently
+ The <literal>Complete subfield (2)</literal> is a reminiscent
+ from the happy &acro.marc;
+ binary format days. &zebra; does not support it, but maps silently
to <literal>Complete field (3)</literal>.
</para>
<note>
- The exact mapping between PQF queries and Zebra internal indexes
- and index types is explained in
+ <para>
+ The exact mapping between &acro.pqf; queries and &zebra; internal indexes
+ and index types is explained in
<xref linkend="querymodel-pqf-apt-mapping"/>.
- </note>
- </sect3>
- </sect2>
-
- </sect1>
+ </para>
+ </note>
+ </section>
+ </section>
- <sect1 id="querymodel-zebra">
- <title>Advanced Zebra PQF Features</title>
+ </section>
+
+ <section id="querymodel-zebra">
+ <title>Extended &zebra; &acro.rpn; Features</title>
<para>
- The Zebra internal query engine has been extended to specific needs
+ The &zebra; internal query engine has been extended to specific needs
not covered by the <literal>bib-1</literal> attribute set query
model. These extensions are <emphasis>non-standard</emphasis>
and <emphasis>non-portable</emphasis>: most functional extensions
are modeled over the <literal>bib-1</literal> attribute set,
- defining type 7-9 attributes.
+ defining type 7 and higher values.
There are also the special
<literal>string</literal> type index names for the
<literal>idxpath</literal> attribute set.
</para>
- <sect2 id="querymodel-zebra-attr-allrecords">
- <title>Zebra specific retrieval of all records</title>
+ <section id="querymodel-zebra-attr-allrecords">
+ <title>&zebra; specific retrieval of all records</title>
<para>
- Zebra defines a hardwired <literal>string</literal> index name
+ &zebra; defines a hardwired <literal>string</literal> index name
called <literal>_ALLRECORDS</literal>. It matches any record
contained in the database, if used in conjunction with
the relation attribute
</screen>
</para>
<warning>
- The special string index <literal>_ALLRECORDS</literal> is
- experimental, and the provided functionality and syntax may very
- well change in future releases of Zebra.
+ <para>
+ The special string index <literal>_ALLRECORDS</literal> is
+ experimental, and the provided functionality and syntax may very
+ well change in future releases of &zebra;.
+ </para>
</warning>
-
- </sect2>
+ </section>
- <sect2 id="querymodel-zebra-attr-search">
- <title>Zebra specific Search Extensions to all Attribute Sets</title>
+ <section id="querymodel-zebra-attr-search">
+ <title>&zebra; specific Search Extensions to all Attribute Sets</title>
<para>
- Zebra extends the Bib1 attribute types, and these extensions are
+ &zebra; extends the &acro.bib1; attribute types, and these extensions are
recognized regardless of attribute
set used in a <literal>search</literal> operation query.
</para>
-
- <table id="querymodel-zebra-attr-search-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Zebra Search Attribute Extensions</caption>
- <thead>
- <tr>
- <td>Name</td>
- <td>Value</td>
- <td>Operation</td>
- <td>Zebra version</td>
- </tr>
+
+ <table id="querymodel-zebra-attr-search-table" frame="top">
+ <title>&zebra; Search Attribute Extensions</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Value</entry>
+ <entry>Operation</entry>
+ <entry>&zebra; version</entry>
+ </row>
</thead>
- <tbody>
- <tr>
- <td>Embedded Sort</td>
- <td>7</td>
- <td>search</td>
- <td>1.1</td>
- </tr>
- <tr>
- <td>Term Set</td>
- <td>8</td>
- <td>search</td>
- <td>1.1</td>
- </tr>
- <tr>
- <td>Rank Weight</td>
- <td>9</td>
- <td>search</td>
- <td>1.1</td>
- </tr>
- <tr>
- <td>Approx Limit</td>
- <td>9</td>
- <td>search</td>
- <td>1.4</td>
- </tr>
- <tr>
- <td>Term Reference</td>
- <td>10</td>
- <td>search</td>
- <td>1.4</td>
- </tr>
- </tbody>
- </table>
-
- <sect3 id="querymodel-zebra-attr-sorting">
- <title>Zebra Extension Embedded Sort Attribute (type 7)</title>
- </sect3>
- <para>
- The embedded sort is a way to specify sort within a query - thus
- removing the need to send a Sort Request separately. It is both
- faster and does not require clients to deal with the Sort
- Facility.
- </para>
-
- <para>
- All ordering operations are based on a lexicographical ordering,
- <emphasis>expect</emphasis> when the
- <literal>structure attribute numeric (109)</literal> is used. In
- this case, ordering is numerical. See
+ <tbody>
+ <row>
+ <entry>Embedded Sort</entry>
+ <entry>7</entry>
+ <entry>search</entry>
+ <entry>1.1</entry>
+ </row>
+ <row>
+ <entry>Term Set</entry>
+ <entry>8</entry>
+ <entry>search</entry>
+ <entry>1.1</entry>
+ </row>
+ <row>
+ <entry>Rank Weight</entry>
+ <entry>9</entry>
+ <entry>search</entry>
+ <entry>1.1</entry>
+ </row>
+ <row>
+ <entry>Term Reference</entry>
+ <entry>10</entry>
+ <entry>search</entry>
+ <entry>1.4</entry>
+ </row>
+ <row>
+ <entry>Local Approx Limit</entry>
+ <entry>11</entry>
+ <entry>search</entry>
+ <entry>1.4</entry>
+ </row>
+ <row>
+ <entry>Global Approx Limit</entry>
+ <entry>12</entry>
+ <entry>search</entry>
+ <entry>2.0.8</entry>
+ </row>
+ <row>
+ <entry>Maximum number of truncated terms (truncmax)</entry>
+ <entry>13</entry>
+ <entry>search</entry>
+ <entry>2.0.10</entry>
+ </row>
+ <row>
+ <entry>
+ Specifies whether un-indexed fields should be ignored.
+ A zero value (default) throws a diagnostic when an un-indexed
+ field is specified. A non-zero value makes it return 0 hits.
+ </entry>
+ <entry>14</entry>
+ <entry>search</entry>
+ <entry>2.0.16</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <section id="querymodel-zebra-attr-sorting">
+ <title>&zebra; Extension Embedded Sort Attribute (type 7)</title>
+ <para>
+ The embedded sort is a way to specify sort within a query - thus
+ removing the need to send a Sort Request separately. It is both
+ faster and does not require clients to deal with the Sort
+ Facility.
+ </para>
+
+ <para>
+ All ordering operations are based on a lexicographical ordering,
+ <emphasis>except</emphasis> when the
+ <literal>structure attribute numeric (109)</literal> is used. In
+ this case, ordering is numerical. See
<xref linkend="querymodel-bib1-structure"/>.
- </para>
+ </para>
+
+ <para>
+ The possible values after attribute <literal>type 7</literal> are
+ <literal>1</literal> ascending and
+ <literal>2</literal> descending.
+ The attributes+term (&acro.apt;) node is separate from the
+ rest and must be <literal>@or</literal>'ed.
+ The term associated with &acro.apt; is the sorting level in integers,
+ where <literal>0</literal> means primary sort,
+ <literal>1</literal> means secondary sort, and so forth.
+ See also <xref linkend="administration-ranking"/>.
+ </para>
+ <para>
+ For example, searching for water, sort by title (ascending)
+ <screen>
+ Z> find @or @attr 1=1016 water @attr 7=1 @attr 1=4 0
+ </screen>
+ </para>
+ <para>
+ Or, searching for water, sort by title ascending, then date descending
+ <screen>
+ Z> find @or @or @attr 1=1016 water @attr 7=1 @attr 1=4 0 @attr 7=2 @attr 1=30 1
+ </screen>
+ </para>
+ </section>
- <para>
- The possible values after attribute <literal>type 7</literal> are
- <literal>1</literal> ascending and
- <literal>2</literal> descending.
- The attributes+term (APT) node is separate from the
- rest and must be <literal>@or</literal>'ed.
- The term associated with APT is the sorting level in integers,
- where <literal>0</literal> means primary sort,
- <literal>1</literal> means secondary sort, and so forth.
- See also <xref linkend="administration-ranking"/>.
- </para>
- <para>
- For example, searching for water, sort by title (ascending)
- <screen>
- Z> find @or @attr 1=1016 water @attr 7=1 @attr 1=4 0
- </screen>
- </para>
- <para>
- Or, searching for water, sort by title ascending, then date descending
- <screen>
- Z> find @or @or @attr 1=1016 water @attr 7=1 @attr 1=4 0 @attr 7=2 @attr 1=30 1
- </screen>
- </para>
+ <!--
+ &zebra; Extension Term Set Attribute
+ From the manual text, I can not see what is the point with this feature.
+ I think it makes more sense when there are multiple terms in a query, or
+ something...
- <sect3 id="querymodel-zebra-attr-estimation">
- <title>Zebra Extension Term Set Attribute (type 8)</title>
- </sect3>
+ We decided 2006-06-03 to disable this feature, as it is covered by
+ scan within a resultset. Better use ressources to upgrade this
+ feature for good performance.
+ -->
+
+ <!--
+ <section id="querymodel-zebra-attr-estimation">
+ <title>&zebra; Extension Term Set Attribute (type 8)</title>
<para>
The Term Set feature is a facility that allows a search to store
hitting terms in a "pseudo" resultset; thus a search (as usual) +
a scan-like facility. Requires a client that can do named result
sets since the search generates two result sets. The value for
attribute 8 is the name of a result set (string). The terms in
- the named term set are returned as SUTRS records.
+ the named term set are returned as &acro.sutrs; records.
</para>
<para>
For example, searching for u in title, right truncated, and
The model has one serious flaw: we don't know the size of term
set. Experimental. Do not use in production code.
</warning>
+ </section>
+ -->
- <sect3 id="querymodel-zebra-attr-weight">
- <title>Zebra Extension Rank Weight Attribute (type 9)</title>
- </sect3>
- <para>
- Rank weight is a way to pass a value to a ranking algorithm - so
- that one APT has one value - while another as a different one.
- See also <xref linkend="administration-ranking"/>.
- </para>
- <para>
- For example, searching for utah in title with weight 30 as well
- as any with weight 20:
- <screen>
- Z> find @attr 2=102 @or @attr 9=30 @attr 1=4 utah @attr 9=20 utah
- </screen>
- </para>
- <sect3 id="querymodel-zebra-attr-limit">
- <title>Zebra Extension Approximative Limit Attribute (type 9)</title>
- </sect3>
- <para>
- Newer Zebra versions normally estimate hit count for every APT
- (leaf) in the query tree. These hit counts are returned as part of
- the searchResult-1 facility in the binary encoded Z39.50 search
- response packages.
- </para>
- <para>
- By setting a limit for the APT we can make Zebra turn into
- approximate hit count when a certain hit count limit is
- reached. A value of zero means exact hit count.
- </para>
- <para>
- For example, we might be interested in exact hit count for a, but
- for b we allow hit count estimates for 1000 and higher.
- <screen>
- Z> find @and a @attr 9=1000 b
- </screen>
- </para>
- <note>
- The estimated hit count facility makes searches faster, as one
- only needs to process large hit lists partially.
- </note>
- <warning>
- This facility clashes with rank weight, because there all
- documents in the hit lists need to be examined for scoring and
- re-sorting.
- It is an experimental
- extension. Do not use in production code.
- </warning>
+ <section id="querymodel-zebra-attr-weight">
+ <title>&zebra; Extension Rank Weight Attribute (type 9)</title>
+ <para>
+ Rank weight is a way to pass a value to a ranking algorithm - so
+ that one &acro.apt; has one value - while another as a different one.
+ See also <xref linkend="administration-ranking"/>.
+ </para>
+ <para>
+ For example, searching for utah in title with weight 30 as well
+ as any with weight 20:
+ <screen>
+ Z> find @attr 2=102 @or @attr 9=30 @attr 1=4 utah @attr 9=20 utah
+ </screen>
+ </para>
+ </section>
- <sect3 id="querymodel-zebra-attr-termref">
- <title>Zebra Extension Term Reference Attribute (type 10)</title>
- </sect3>
- <para>
- Zebra supports the <literal>searchResult-1</literal> facility.
- If the <literal>Term Reference Attribute (type 10)</literal> is
- given, that specifies a subqueryId value returned as part of the
- search result. It is a way for a client to name an APT part of a
- query.
- </para>
- <!--
- <para>
- <screen>
- </screen>
- </para>
- -->
- <warning>
- Experimental. Do not use in production code.
- </warning>
+ <section id="querymodel-zebra-attr-termref">
+ <title>&zebra; Extension Term Reference Attribute (type 10)</title>
+ <para>
+ &zebra; supports the searchResult-1 facility.
+ If the Term Reference Attribute (type 10) is
+ given, that specifies a subqueryId value returned as part of the
+ search result. It is a way for a client to name an &acro.apt; part of a
+ query.
+ </para>
+ <warning>
+ <para>
+ Experimental. Do not use in production code.
+ </para>
+ </warning>
+
+ </section>
+
+
+
+ <section id="querymodel-zebra-local-attr-limit">
+ <title>Local Approximative Limit Attribute (type 11)</title>
+ <para>
+ &zebra; computes - unless otherwise configured -
+ the exact hit count for every &acro.apt;
+ (leaf) in the query tree. These hit counts are returned as part of
+ the searchResult-1 facility in the binary encoded &acro.z3950; search
+ response packages.
+ </para>
+ <para>
+ By setting an estimation limit size of the resultset of the &acro.apt;
+ leaves, &zebra; stops processing the result set when the limit
+ length is reached.
+ Hit counts under this limit are still precise, but hit counts over it
+ are estimated using the statistics gathered from the chopped
+ result set.
+ </para>
+ <para>
+ Specifying a limit of <literal>0</literal> results in exact hit counts.
+ </para>
+ <para>
+ For example, we might be interested in exact hit count for a, but
+ for b we allow hit count estimates for 1000 and higher.
+ <screen>
+ Z> find @and a @attr 11=1000 b
+ </screen>
+ </para>
+ <note>
+ <para>
+ The estimated hit count facility makes searches faster, as one
+ only needs to process large hit lists partially.
+ It is mostly used in huge databases, where you you want trade
+ exactness of hit counts against speed of execution.
+ </para>
+ </note>
+ <warning>
+ <para>
+ Do not use approximative hit count limits
+ in conjunction with relevance ranking, as re-sorting of the
+ result set only works when the entire result set has
+ been processed.
+ </para>
+ </warning>
+ </section>
- </sect2>
-
+ <section id="querymodel-zebra-global-attr-limit">
+ <title>Global Approximative Limit Attribute (type 12)</title>
+ <para>
+ By default &zebra; computes precise hit counts for a query as
+ a whole. Setting attribute 12 makes it perform approximative
+ hit counts instead. It has the same semantics as
+ <literal>estimatehits</literal> for the <xref linkend="zebra-cfg"/>.
+ </para>
+ <para>
+ The attribute (12) can occur anywhere in the query tree.
+ Unlike regular attributes it does not relate to the leaf (&acro.apt;)
+ - but to the whole query.
+ </para>
+ <warning>
+ <para>
+ Do not use approximative hit count limits
+ in conjunction with relevance ranking, as re-sorting of the
+ result set only works when the entire result set has
+ been processed.
+ </para>
+ </warning>
+ </section>
- <sect2 id="querymodel-zebra-attr-scan">
- <title>Zebra specific Scan Extensions to all Attribute Sets</title>
+ </section>
+
+ <section id="querymodel-zebra-attr-scan">
+ <title>&zebra; specific Scan Extensions to all Attribute Sets</title>
<para>
- Zebra extends the Bib1 attribute types, and these extensions are
+ &zebra; extends the Bib1 attribute types, and these extensions are
recognized regardless of attribute
- set used in a <literal>scan</literal> operation query.
+ set used in a scan operation query.
</para>
- <table id="querymodel-zebra-attr-scan-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Zebra Scan Attribute Extensions</caption>
- <thead>
- <tr>
- <td>Name</td>
- <td>Type</td>
- <td>Operation</td>
- <td>Zebra version</td>
- </tr>
+ <table id="querymodel-zebra-attr-scan-table" frame="top">
+ <title>&zebra; Scan Attribute Extensions</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Type</entry>
+ <entry>Operation</entry>
+ <entry>&zebra; version</entry>
+ </row>
</thead>
- <tbody>
- <tr>
- <td>Result Set Narrow</td>
- <td>8</td>
- <td>scan</td>
- <td>1.3</td>
- </tr>
- <tr>
- <td>Approximative Limit</td>
- <td>9</td>
- <td>scan</td>
- <td>1.4</td>
- </tr>
- </tbody>
- </table>
-
- <sect3 id="querymodel-zebra-attr-narrow">
- <title>Zebra Extension Result Set Narrow (type 8)</title>
- </sect3>
- <para>
- If attribute <literal>Result Set Narrow (type 8)</literal>
- is given for <literal>scan</literal>, the value is the name of a
- result set. Each hit count in <literal>scan</literal> is
- <literal>@and</literal>'ed with the result set given.
- </para>
- <para>
- Consider for example
- the case of scanning all title fields around the
- scanterm <emphasis>mozart</emphasis>, then refining the scan by
- issuing a filtering query for <emphasis>amadeus</emphasis> to
- restrict the scan to the result set of the query:
- <screen>
+ <tbody>
+ <row>
+ <entry>Result Set Narrow</entry>
+ <entry>8</entry>
+ <entry>scan</entry>
+ <entry>1.3</entry>
+ </row>
+ <row>
+ <entry>Approximative Limit</entry>
+ <entry>12</entry>
+ <entry>scan</entry>
+ <entry>2.0.20</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <section id="querymodel-zebra-attr-narrow">
+ <title>&zebra; Extension Result Set Narrow (type 8)</title>
+ <para>
+ If attribute Result Set Narrow (type 8)
+ is given for scan, the value is the name of a
+ result set. Each hit count in scan is
+ <literal>@and</literal>'ed with the result set given.
+ </para>
+ <para>
+ Consider for example
+ the case of scanning all title fields around the
+ scanterm <emphasis>mozart</emphasis>, then refining the scan by
+ issuing a filtering query for <emphasis>amadeus</emphasis> to
+ restrict the scan to the result set of the query:
+ <screen>
Z> scan @attr 1=4 mozart
...
* mozart (43)
mozartiana (0)
mozarts (1)
...
- </screen>
- </para>
-
- <warning>
- Experimental. Do not use in production code.
- </warning>
-
- <sect3 id="querymodel-zebra-attr-approx">
- <title>Zebra Extension Approximative Limit (type 9)</title>
- </sect3>
- <para>
- The <literal>Zebra Extension Approximative Limit (type
- 9)</literal> is a way to enable approximate
- hit counts for <literal>scan</literal> hit counts, in the same
- way as for <literal>search</literal> hit counts.
- </para>
- <!--
- <para>
- <screen>
- </screen>
- </para>
- -->
- <warning>
- Experimental and buggy. Definitely not to be used in production code.
- </warning>
-
+ </screen>
+ </para>
+
+ <para>
+ &zebra; 2.0.2 and later is able to skip 0 hit counts. This, however,
+ is known not to scale if the number of terms to skip is high.
+ This most likely will happen if the result set is small (and
+ result in many 0 hits).
+ </para>
+ </section>
- </sect2>
-
+ <section id="querymodel-zebra-attr-approx">
+ <title>&zebra; Extension Approximative Limit (type 12)</title>
+ <para>
+ The &zebra; Extension Approximative Limit (type 12) is a way to
+ enable approximate hit counts for scan hit counts, in the same
+ way as for search hit counts.
+ </para>
+ </section>
+ </section>
- <sect2 id="querymodel-idxpath">
- <title>Zebra special IDXPATH Attribute Set for GRS indexing</title>
+ <section id="querymodel-idxpath">
+ <title>&zebra; special &acro.idxpath; Attribute Set for &acro.grs1; indexing</title>
<para>
The attribute-set <literal>idxpath</literal> consists of a single
- <literal>Use (type 1)</literal> attribute. All non-use attributes
- behave as normal.
+ Use (type 1) attribute. All non-use attributes behave as normal.
</para>
<para>
This feature is enabled when defining the
- <literal>xpath enable</literal> option in the GRS filter
+ <literal>xpath enable</literal> option in the &acro.grs1; filter
<filename>*.abs</filename> configuration files. If one wants to use
the special <literal>idxpath</literal> numeric attribute set, the
- main Zebra configuration file <filename>zebra.cfg</filename>
+ main &zebra; configuration file <filename>zebra.cfg</filename>
directive <literal>attset: idxpath.att</literal> must be enabled.
</para>
- <warning>The <literal>idxpath</literal> is depreciated, may not be
- supported in future Zebra versions, and should definitely
- not be used in production code.
+ <warning>
+ <para>
+ The <literal>idxpath</literal> is deprecated, may not be
+ supported in future &zebra; versions, and should definitely
+ not be used in production code.
+ </para>
</warning>
- <sect3 id="querymodel-idxpath-use">
- <title>IDXPATH Use Attributes (type = 1)</title>
+ <section id="querymodel-idxpath-use">
+ <title>&acro.idxpath; Use Attributes (type = 1)</title>
<para>
- This attribute set allows one to search GRS filter indexed
- records by XPATH like structured index names.
+ This attribute set allows one to search &acro.grs1; filter indexed
+ records by &acro.xpath; like structured index names.
</para>
- <warning>The <literal>idxpath</literal> option defines hard-coded
- index names, which might clash with your own index names.
+ <warning>
+ <para>
+ The <literal>idxpath</literal> option defines hard-coded
+ index names, which might clash with your own index names.
+ </para>
</warning>
- <table id="querymodel-idxpath-use-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Zebra specific IDXPATH Use Attributes (type 1)</caption>
- <thead>
- <tr>
- <td>IDXPATH</td>
- <td>Value</td>
- <td>String Index</td>
- <td>Notes</td>
- </tr>
+ <table id="querymodel-idxpath-use-table" frame="top">
+ <title>&zebra; specific &acro.idxpath; Use Attributes (type 1)</title>
+ <tgroup cols="4">
+ <thead>
+ <row>
+ <entry>&acro.idxpath;</entry>
+ <entry>Value</entry>
+ <entry>String Index</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td>XPATH Begin</td>
- <td>1</td>
- <td>_XPATH_BEGIN</td>
- <td>depreciated</td>
- </tr>
- <tr>
- <td>XPATH End</td>
- <td>2</td>
- <td>_XPATH_END</td>
- <td>depreciated</td>
- </tr>
- <tr>
- <td>XPATH CData</td>
- <td>1016</td>
- <td>_XPATH_CDATA</td>
- <td>depreciated</td>
- </tr>
- <tr>
- <td>XPATH Attribute Name</td>
- <td>3</td>
- <td>_XPATH_ATTR_NAME</td>
- <td>depreciated</td>
- </tr>
- <tr>
- <td>XPATH Attribute CData</td>
- <td>1015</td>
- <td>_XPATH_ATTR_CDATA</td>
- <td>depreciated</td>
- </tr>
+ <row>
+ <entry>&acro.xpath; Begin</entry>
+ <entry>1</entry>
+ <entry>_XPATH_BEGIN</entry>
+ <entry>deprecated</entry>
+ </row>
+ <row>
+ <entry>&acro.xpath; End</entry>
+ <entry>2</entry>
+ <entry>_XPATH_END</entry>
+ <entry>deprecated</entry>
+ </row>
+ <row>
+ <entry>&acro.xpath; CData</entry>
+ <entry>1016</entry>
+ <entry>_XPATH_CDATA</entry>
+ <entry>deprecated</entry>
+ </row>
+ <row>
+ <entry>&acro.xpath; Attribute Name</entry>
+ <entry>3</entry>
+ <entry>_XPATH_ATTR_NAME</entry>
+ <entry>deprecated</entry>
+ </row>
+ <row>
+ <entry>&acro.xpath; Attribute CData</entry>
+ <entry>1015</entry>
+ <entry>_XPATH_ATTR_CDATA</entry>
+ <entry>deprecated</entry>
+ </row>
</tbody>
+ </tgroup>
</table>
-
<para>
See <filename>tab/idxpath.att</filename> for more information.
</para>
</screen>
</para>
<para>
- Search for all documents where specific nested XPATH
+ Search for all documents where specific nested &acro.xpath;
<literal>/c1/c2/../cn</literal> exists. Notice the very
counter-intuitive <emphasis>reverse</emphasis> notation!
<screen>
</screen>
</para>
<para>
- Search for all documents with have an XML element node
- including an XML attribute named <emphasis>creator</emphasis>
+ Search for all documents with have an &acro.xml; element node
+ including an &acro.xml; attribute named <emphasis>creator</emphasis>
<screen>
Z> find @attrset idxpath @attr 1=3 @attr 4=3 creator
Z> find @attr 1=_XPATH_ATTR_NAME @attr 4=3 creator
</screen>
</para>
- </sect3>
- </sect2>
+ </section>
+ </section>
- <sect2 id="querymodel-pqf-apt-mapping">
- <title>Mapping from PQF atomic APT queries to Zebra internal
+ <section id="querymodel-pqf-apt-mapping">
+ <title>Mapping from &acro.pqf; atomic &acro.apt; queries to &zebra; internal
register indexes</title>
<para>
- The rules for PQF APT mapping are rather tricky to grasp in the
+ The rules for &acro.pqf; &acro.apt; mapping are rather tricky to grasp in the
first place. We deal first with the rules for deciding which
internal register or string index to use, according to the use
attribute or access point specified in the query. Thereafter we
the named register.
</para>
- <sect3 id="querymodel-pqf-apt-mapping-accesspoint">
- <title>Mapping of PQF APT access points</title>
+ <section id="querymodel-pqf-apt-mapping-accesspoint">
+ <title>Mapping of &acro.pqf; &acro.apt; access points</title>
<para>
- Zebra understands four fundamental different types of access
+ &zebra; understands four fundamental different types of access
points, of which only the
<emphasis>numeric use attribute</emphasis> type access points
- are defined by the <ulink url="&url.z39.50;">Z39.50</ulink>
+ are defined by the <ulink url="&url.z39.50;">&acro.z3950;</ulink>
standard.
- All other access point types are Zebra specific, and non-portable.
+ All other access point types are &zebra; specific, and non-portable.
</para>
- <table id="querymodel-zebra-mapping-accesspoint-types"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Access point name mapping</caption>
+ <table id="querymodel-zebra-mapping-accesspoint-types" frame="top">
+ <title>Access point name mapping</title>
+ <tgroup cols="4">
<thead>
- <tr>
- <td>Access Point</td>
- <td>Type</td>
- <td>Grammar</td>
- <td>Notes</td>
- </tr>
+ <row>
+ <entry>Access Point</entry>
+ <entry>Type</entry>
+ <entry>Grammar</entry>
+ <entry>Notes</entry>
+ </row>
</thead>
<tbody>
- <tr>
- <td>Use attribute</td>
- <td>numeric</td>
- <td>[1-9][1-9]*</td>
- <td>directly mapped to string index name</td>
- </tr>
- <tr>
- <td>String index name</td>
- <td>string</td>
- <td>[a-zA-Z](\-?[a-zA-Z0-9])*</td>
- <td>normalized name is used as internal string index name</td>
- </tr>
- <tr>
- <td>Zebra internal index name</td>
- <td>zebra</td>
- <td>_[a-zA-Z](_?[a-zA-Z0-9])*</td>
- <td>hardwired internal string index name</td>
- </tr>
- <tr>
- <td>XPATH special index</td>
- <td>XPath</td>
- <td>/.*</td>
- <td>special xpath search for GRS indexed records</td>
- </tr>
- </tbody>
- </table>
-
- <para>
- <literal>Attribute set names</literal> and
- <literal>string index names</literal> are normalizes
- according to the following rules: all <emphasis>single</emphasis>
- hyphens <literal>'-'</literal> are stripped, and all upper case
- letters are folded to lower case.
+ <row>
+ <entry>Use attribute</entry>
+ <entry>numeric</entry>
+ <entry>[1-9][1-9]*</entry>
+ <entry>directly mapped to string index name</entry>
+ </row>
+ <row>
+ <entry>String index name</entry>
+ <entry>string</entry>
+ <entry>[a-zA-Z](\-?[a-zA-Z0-9])*</entry>
+ <entry>normalized name is used as internal string index name</entry>
+ </row>
+ <row>
+ <entry>&zebra; internal index name</entry>
+ <entry>zebra</entry>
+ <entry>_[a-zA-Z](_?[a-zA-Z0-9])*</entry>
+ <entry>hardwired internal string index name</entry>
+ </row>
+ <row>
+ <entry>&acro.xpath; special index</entry>
+ <entry>XPath</entry>
+ <entry>/.*</entry>
+ <entry>special xpath search for &acro.grs1; indexed records</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ <literal>Attribute set names</literal> and
+ <literal>string index names</literal> are normalizes
+ according to the following rules: all <emphasis>single</emphasis>
+ hyphens <literal>'-'</literal> are stripped, and all upper case
+ letters are folded to lower case.
</para>
-
+
<para>
<emphasis>Numeric use attributes</emphasis> are mapped
- to the Zebra internal
+ to the &zebra; internal
string index according to the attribute set definition in use.
- The default attribute set is <literal>Bib-1</literal>, and may be
- omitted in the PQF query.
+ The default attribute set is &acro.bib1;, and may be
+ omitted in the &acro.pqf; query.
</para>
<para>
According to normalization and numeric
use attribute mapping, it follows that the following
- PQF queries are considered equivalent (assuming the default
+ &acro.pqf; queries are considered equivalent (assuming the default
configuration has not been altered):
<screen>
Z> find @attr 1=Body-of-text serenade
Z> find @attr 1=BodyOfText serenade
Z> find @attr 1=bO-d-Y-of-tE-x-t serenade
Z> find @attr 1=1010 serenade
- Z> find @attrset Bib-1 @attr 1=1010 serenade
+ Z> find @attrset bib1 @attr 1=1010 serenade
Z> find @attrset bib1 @attr 1=1010 serenade
Z> find @attrset Bib1 @attr 1=1010 serenade
Z> find @attrset b-I-b-1 @attr 1=1010 serenade
fields as specified in the <literal>.abs</literal> file which
describes the profile of the records which have been loaded.
If no use attribute is provided, a default of
- <literal>Bib-1 Use Any (1016)</literal> is
- assumed.
- The predefined <literal>use attribute sets</literal>
+ &acro.bib1; Use Any (1016) is assumed.
+ The predefined use attribute sets
can be reconfigured by tweaking the configuration files
<filename>tab/*.att</filename>, and
new attribute sets can be defined by adding similar files in the
</para>
<para>
- <literal>String indexes</literal> can be accessed directly,
+ String indexes can be accessed directly,
independently which attribute set is in use. These are just
ignored. The above mentioned name normalization applies.
- <literal>String index names</literal> are defined in the
+ String index names are defined in the
used indexing filter configuration files, for example in the
- <literal>GRS</literal>
+ &acro.grs1;
<filename>*.abs</filename> configuration files, or in the
- <literal>alvis</literal> filter XSLT indexing stylesheets.
+ <literal>alvis</literal> filter &acro.xslt; indexing stylesheets.
</para>
<para>
- <literal>Zebra internal indexes</literal> can be accessed directly,
+ &zebra; internal indexes can be accessed directly,
according to the same rules as the user defined
- <literal>string indexes</literal>. The only difference is that
- <literal>Zebra internal index names</literal> are hardwired,
+ string indexes. The only difference is that
+ &zebra; internal index names are hardwired,
all uppercase and
must start with the character <literal>'_'</literal>.
</para>
<para>
- Finally, <literal>XPATH</literal> access points are only
- available using the <literal>GRS</literal> filter for indexing.
+ Finally, &acro.xpath; access points are only
+ available using the &acro.grs1; filter for indexing.
These access point names must start with the character
<literal>'/'</literal>, they are <emphasis>not
- normalized</emphasis>, but passed unaltered to the Zebra internal
- XPATH engine. See <xref linkend="querymodel-use-xpath"/>.
+ normalized</emphasis>, but passed unaltered to the &zebra; internal
+ &acro.xpath; engine. See <xref linkend="querymodel-use-xpath"/>.
</para>
- </sect3>
+ </section>
- <sect3 id="querymodel-pqf-apt-mapping-structuretype">
- <title>Mapping of PQF APT structure and completeness to
+ <section id="querymodel-pqf-apt-mapping-structuretype">
+ <title>Mapping of &acro.pqf; &acro.apt; structure and completeness to
register type</title>
<para>
- Internally Zebra has in it's default configuration several
+ Internally &zebra; has in its default configuration several
different types of registers or indexes, whose tokenization and
character normalization rules differ. This reflects the fact that
searching fundamental different tokens like dates, numbers,
bitfields and string based text needs different rule sets.
</para>
- <table id="querymodel-zebra-mapping-structure-types"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Structure and completeness mapping to register types</caption>
+ <table id="querymodel-zebra-mapping-structure-types" frame="top">
+ <title>Structure and completeness mapping to register types</title>
+ <tgroup cols="4">
<thead>
- <tr>
- <td>Structure</td>
- <td>Completeness</td>
- <td>Register type</td>
- <td>Notes</td>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>
+ <row>
+ <entry>Structure</entry>
+ <entry>Completeness</entry>
+ <entry>Register type</entry>
+ <entry>Notes</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
phrase (@attr 4=1), word (@attr 4=2),
word-list (@attr 4=6),
free-form-text (@attr 4=105), or document-text (@attr 4=106)
- </td>
- <td>Incomplete field (@attr 6=1)</td>
- <td>Word ('w')</td>
- <td>Traditional tokenized and character normalized word index</td>
- </tr>
- <tr>
- <td>
+ </entry>
+ <entry>Incomplete field (@attr 6=1)</entry>
+ <entry>Word ('w')</entry>
+ <entry>Traditional tokenized and character normalized word index</entry>
+ </row>
+ <row>
+ <entry>
phrase (@attr 4=1), word (@attr 4=2),
word-list (@attr 4=6),
free-form-text (@attr 4=105), or document-text (@attr 4=106)
- </td>
- <td>complete field' (@attr 6=3)</td>
- <td>Phrase ('p')</td>
- <td>Character normalized, but not tokenized index for phrase
+ </entry>
+ <entry>complete field' (@attr 6=3)</entry>
+ <entry>Phrase ('p')</entry>
+ <entry>Character normalized, but not tokenized index for phrase
matches
- </td>
- </tr>
- <tr>
- <td>urx (@attr 4=104)</td>
- <td>ignored</td>
- <td>URX/URL ('u')</td>
- <td>Special index for URL web addresses</td>
- </tr>
- <tr>
- <td>numeric (@attr 4=109)</td>
- <td>ignored</td>
- <td>Numeric ('u')</td>
- <td>Special index for digital numbers</td>
- </tr>
- <tr>
- <td>key (@attr 4=3)</td>
- <td>ignored</td>
- <td>Null bitmap ('0')</td>
- <td>Used for non-tokenizated and non-normalized bit sequences</td>
- </tr>
- <tr>
- <td>year (@attr 4=4)</td>
- <td>ignored</td>
- <td>Year ('y')</td>
- <td>Non-tokenizated and non-normalized 4 digit numbers</td>
- </tr>
- <tr>
- <td>date (@attr 4=5)</td>
- <td>ignored</td>
- <td>Date ('d')</td>
- <td>Non-tokenizated and non-normalized ISO date strings</td>
- </tr>
- <tr>
- <td>ignored</td>
- <td>ignored</td>
- <td>Sort ('s')</td>
- <td>Used with special sort attribute set (@attr 7=1, @attr 7=2)</td>
- </tr>
- <tr>
- <td>overruled</td>
- <td>overruled</td>
- <td>special</td>
- <td>Internal record ID register, used whenever
- Relation Always Matches (@attr 2=103) is specified</td>
- </tr>
- </tbody>
- </table>
-
+ </entry>
+ </row>
+ <row>
+ <entry>urx (@attr 4=104)</entry>
+ <entry>ignored</entry>
+ <entry>URX/URL ('u')</entry>
+ <entry>Special index for URL web addresses</entry>
+ </row>
+ <row>
+ <entry>numeric (@attr 4=109)</entry>
+ <entry>ignored</entry>
+ <entry>Numeric ('n')</entry>
+ <entry>Special index for digital numbers</entry>
+ </row>
+ <row>
+ <entry>key (@attr 4=3)</entry>
+ <entry>ignored</entry>
+ <entry>Null bitmap ('0')</entry>
+ <entry>Used for non-tokenized and non-normalized bit sequences</entry>
+ </row>
+ <row>
+ <entry>year (@attr 4=4)</entry>
+ <entry>ignored</entry>
+ <entry>Year ('y')</entry>
+ <entry>Non-tokenized and non-normalized 4 digit numbers</entry>
+ </row>
+ <row>
+ <entry>date (@attr 4=5)</entry>
+ <entry>ignored</entry>
+ <entry>Date ('d')</entry>
+ <entry>Non-tokenized and non-normalized ISO date strings</entry>
+ </row>
+ <row>
+ <entry>ignored</entry>
+ <entry>ignored</entry>
+ <entry>Sort ('s')</entry>
+ <entry>Used with special sort attribute set (@attr 7=1, @attr 7=2)</entry>
+ </row>
+ <row>
+ <entry>overruled</entry>
+ <entry>overruled</entry>
+ <entry>special</entry>
+ <entry>Internal record ID register, used whenever
+ Relation Always Matches (@attr 2=103) is specified</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
<!-- see in util/zebramap.c -->
<para>
against the contents of the phrase (long word) register, if one
exists for the given <emphasis>Use</emphasis> attribute.
A phrase register is created for those fields in the
- GRS <filename>*.abs</filename> file that contains a
+ &acro.grs1; <filename>*.abs</filename> file that contains a
<literal>p</literal>-specifier.
<screen>
- Z> scan @attr 1=Title @attr 4=1 @attr 6=3 beethoven
+ Z> scan @attr 1=Title @attr 4=1 @attr 6=3 beethoven
...
bayreuther festspiele (1)
* beethoven bibliography database (1)
contains multiple words, the term will only match if all of the words
are found immediately adjacent, and in the given order.
The word search is performed on those fields that are indexed as
- type <literal>w</literal> in the GRS <filename>*.abs</filename> file.
+ type <literal>w</literal> in the &acro.grs1; <filename>*.abs</filename> file.
<screen>
- Z> scan @attr 1=Title @attr 4=1 @attr 6=1 beethoven
+ Z> scan @attr 1=Title @attr 4=1 @attr 6=1 beethoven
...
beefheart (1)
* beethoven (18)
natural-language, relevance-ranked query.
This search type uses the word register, i.e. those fields
that are indexed as type <literal>w</literal> in the
- GRS <filename>*.abs</filename> file.
+ &acro.grs1; <filename>*.abs</filename> file.
</para>
<para>
If the <emphasis>Structure</emphasis> attribute is
<emphasis>Numeric String</emphasis> the term is treated as an integer.
The search is performed on those fields that are indexed
- as type <literal>n</literal> in the GRS
+ as type <literal>n</literal> in the &acro.grs1;
<filename>*.abs</filename> file.
</para>
<para>
If the <emphasis>Structure</emphasis> attribute is
<emphasis>Local Number</emphasis> the term is treated as
- native Zebra Record Identifier.
+ native &zebra; Record Identifier.
</para>
<para>
contents.
</para>
- </sect3>
- </sect2>
+ </section>
+ </section>
- <sect2 id="querymodel-regular">
- <title>Zebra Regular Expressions in Truncation Attribute (type = 5)</title>
+ <section id="querymodel-regular">
+ <title>&zebra; Regular Expressions in Truncation Attribute (type = 5)</title>
<para>
Each term in a query is interpreted as a regular expression if
Both query types follow the same syntax with the operands:
</para>
- <table id="querymodel-regular-operands-table"
- frame="all" rowsep="1" colsep="1" align="center">
-
- <caption>Regular Expression Operands</caption>
- <!--
- <thead>
- <tr><td>one</td><td>two</td></tr>
- </thead>
- -->
- <tbody>
- <tr>
- <td><literal>x</literal></td>
- <td>Matches the character <literal>x</literal>.</td>
- </tr>
- <tr>
- <td><literal>.</literal></td>
- <td>Matches any character.</td>
- </tr>
- <tr>
- <td><literal>[ .. ]</literal></td>
- <td>Matches the set of characters specified;
- such as <literal>[abc]</literal> or <literal>[a-c]</literal>.</td>
- </tr>
- </tbody>
- </table>
+ <table id="querymodel-regular-operands-table" frame="top">
+ <title>Regular Expression Operands</title>
+ <tgroup cols="2">
+ <tbody>
+ <row>
+ <entry><literal>x</literal></entry>
+ <entry>Matches the character <literal>x</literal>.</entry>
+ </row>
+ <row>
+ <entry><literal>.</literal></entry>
+ <entry>Matches any character.</entry>
+ </row>
+ <row>
+ <entry><literal>[ .. ]</literal></entry>
+ <entry>Matches the set of characters specified;
+ such as <literal>[abc]</literal> or <literal>[a-c]</literal>.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
<para>
The above operands can be combined with the following operators:
</para>
-
- <table id="querymodel-regular-operators-table"
- frame="all" rowsep="1" colsep="1" align="center">
- <caption>Regular Expression Operators</caption>
- <!--
- <thead>
- <tr><td>one</td><td>two</td></tr>
- </thead>
- -->
- <tbody>
- <tr>
- <td><literal>x*</literal></td>
- <td>Matches <literal>x</literal> zero or more times.
- Priority: high.</td>
- </tr>
- <tr>
- <td><literal>x+</literal></td>
- <td>Matches <literal>x</literal> one or more times.
- Priority: high.</td>
- </tr>
- <tr>
- <td><literal>x?</literal></td>
- <td> Matches <literal>x</literal> zero or once.
- Priority: high.</td>
- </tr>
- <tr>
- <td><literal>xy</literal></td>
- <td> Matches <literal>x</literal>, then <literal>y</literal>.
- Priority: medium.</td>
- </tr>
- <tr>
- <td><literal>x|y</literal></td>
- <td> Matches either <literal>x</literal> or <literal>y</literal>.
- Priority: low.</td>
- </tr>
- <tr>
- <td><literal>( )</literal></td>
- <td>The order of evaluation may be changed by using parentheses.</td>
- </tr>
- </tbody>
- </table>
-
+
+ <table id="querymodel-regular-operators-table" frame="top">
+ <title>Regular Expression Operators</title>
+ <tgroup cols="2">
+ <tbody>
+ <row>
+ <entry><literal>x*</literal></entry>
+ <entry>Matches <literal>x</literal> zero or more times.
+ Priority: high.</entry>
+ </row>
+ <row>
+ <entry><literal>x+</literal></entry>
+ <entry>Matches <literal>x</literal> one or more times.
+ Priority: high.</entry>
+ </row>
+ <row>
+ <entry><literal>x?</literal></entry>
+ <entry> Matches <literal>x</literal> zero or once.
+ Priority: high.</entry>
+ </row>
+ <row>
+ <entry><literal>xy</literal></entry>
+ <entry> Matches <literal>x</literal>, then <literal>y</literal>.
+ Priority: medium.</entry>
+ </row>
+ <row>
+ <entry><literal>x|y</literal></entry>
+ <entry> Matches either <literal>x</literal> or <literal>y</literal>.
+ Priority: low.</entry>
+ </row>
+ <row>
+ <entry><literal>( )</literal></entry>
+ <entry>The order of evaluation may be changed by using parentheses.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
<para>
If the first character of the <literal>Regxp-2</literal> query
is a plus character (<literal>+</literal>) it marks the
beginning of a section with non-standard specifiers.
The next plus character marks the end of the section.
- Currently Zebra only supports one specifier, the error tolerance,
+ Currently &zebra; only supports one specifier, the error tolerance,
which consists one digit.
<!-- TODO Nice thing, but what does
that error tolerance digit *mean*? Maybe an example would be nice? -->
Z> find @attr 1=4 @attr 5=102 @attr 2=102 "informat.* retrieval"
</screen>
</para>
- </sect2>
+ </section>
<!--
<para>
The RecordType parameter in the <literal>zebra.cfg</literal> file, or
- the <literal>-t</literal> option to the indexer tells Zebra how to
+ the <literal>-t</literal> option to the indexer tells &zebra; how to
process input records.
Two basic types of processing are available - raw text and structured
data. Raw text is just that, and it is selected by providing the
- argument <literal>text</literal> to Zebra. Structured records are
+ argument <literal>text</literal> to &zebra;. Structured records are
all handled internally using the basic mechanisms described in the
subsequent sections.
- Zebra can read structured records in many different formats.
+ &zebra; can read structured records in many different formats.
</para>
-->
- </sect1>
+ </section>
- <sect1 id="querymodel-cql-to-pqf">
- <title>Server Side CQL to PQF Query Translation</title>
+ <section id="querymodel-cql-to-pqf">
+ <title>Server Side &acro.cql; to &acro.pqf; Query Translation</title>
<para>
Using the
<literal><cql2rpn>l2rpn.txt</cql2rpn></literal>
- YAZ Frontend Virtual
+ &yaz; Frontend Virtual
Hosts option, one can configure
- the YAZ Frontend CQL-to-PQF
+ the &yaz; Frontend &acro.cql;-to-&acro.pqf;
converter, specifying the interpretation of various
- <ulink url="&url.cql;">CQL</ulink>
+ <ulink url="&url.cql;">&acro.cql;</ulink>
indexes, relations, etc. in terms of Type-1 query attributes.
<!-- The yaz-client config file -->
</para>
<para>
- For example, using server-side CQL-to-PQF conversion, one might
+ For example, using server-side &acro.cql;-to-&acro.pqf; conversion, one might
query a zebra server like this:
<screen>
<![CDATA[
]]>
</screen>
and - if properly configured - even static relevance ranking can
- be performed using CQL query syntax:
+ be performed using &acro.cql; query syntax:
<screen>
<
projects / idzebra-moved-to-github.git / blobdiff