<chapter id="querymodel">
<title>Query Model</title>
-
+
<section id="querymodel-overview">
- <title>Query Model Overview</title>
-
+ <title>Query Model Overview</title>
+
<section id="querymodel-query-languages">
<title>Query Languages</title>
-
+
<para>
&zebra; is born as a networking Information Retrieval engine adhering
- to the international standards
+ to the international standards
<ulink url="&url.z39.50;">&acro.z3950;</ulink> and
<ulink url="&url.sru;">&acro.sru;</ulink>,
- and implement the
+ and implement the
type-1 Reverse Polish Notation (&acro.rpn;) query
model defined there.
Unfortunately, this model has only defined a binary
encoded representation, which is used as transport packaging in
the &acro.z3950; protocol layer. This representation is not human
- readable, nor defines any convenient way to specify queries.
+ readable, nor defines any convenient way to specify queries.
</para>
<para>
Since the type-1 (&acro.rpn;)
representation, every client application needs to provide some
form of mapping from a local query notation or representation to it.
</para>
-
-
+
+
<section id="querymodel-query-languages-pqf">
<title>Prefix Query Format (&acro.pqf;)</title>
<para>
- Index Data has defined a textual representation in the
+ Index Data has defined a textual representation in the
<ulink url="&url.yaz.pqf;">Prefix Query Format</ulink>, short
- <emphasis>&acro.pqf;</emphasis>, which maps
- one-to-one to binary encoded
+ <emphasis>&acro.pqf;</emphasis>, which maps
+ one-to-one to binary encoded
<emphasis>type-1 &acro.rpn;</emphasis> queries.
&acro.pqf; has been adopted by other
parties developing &acro.z3950; software, and is often referred to as
- <emphasis>Prefix Query Notation</emphasis>, or in short
- &acro.pqn;. See
+ <emphasis>Prefix Query Notation</emphasis>, or in short
+ &acro.pqn;. See
<xref linkend="querymodel-rpn"/> for further explanations and
- descriptions of &zebra;'s capabilities.
+ descriptions of &zebra;'s capabilities.
</para>
- </section>
-
+ </section>
+
<section id="querymodel-query-languages-cql">
<title>Common Query Language (&acro.cql;)</title>
<para>
The query model of the type-1 &acro.rpn;,
- expressed in &acro.pqf;/&acro.pqn; is natively supported.
+ expressed in &acro.pqf;/&acro.pqn; is natively supported.
On the other hand, the default &acro.sru;
web services <emphasis>Common Query Language</emphasis>
<ulink url="&url.cql;">&acro.cql;</ulink> is not natively supported.
&zebra; can be configured to understand and map &acro.cql; to &acro.pqf;. See
<xref linkend="querymodel-cql-to-pqf"/>.
</para>
- </section>
-
+ </section>
+
</section>
<section id="querymodel-operation-types">
<para>
&zebra; supports all of the three different
&acro.z3950;/&acro.sru; operations defined in the
- standards: explain, search,
+ standards: explain, search,
and scan. A short description of the
- functionality and purpose of each is quite in order here.
+ functionality and purpose of each is quite in order here.
</para>
<section id="querymodel-operation-type-explain">
well known to any client, but the specific
<emphasis>semantics</emphasis> - taking into account a
particular servers functionalities and abilities - must be
- discovered from case to case. Enters the
- explain operation, which provides the means for learning which
+ discovered from case to case. Enters the
+ explain operation, which provides the means for learning which
<emphasis>fields</emphasis> (also called
<emphasis>indexes</emphasis> or <emphasis>access points</emphasis>)
are provided, which default parameter the server uses, which
retrieve document formats are defined, and which specific parts
- of the general query model are supported.
+ of the general query model are supported.
</para>
<para>
The &acro.z3950; embeds the explain operation
- by performing a
- search in the magic
+ by performing a
+ search in the magic
<literal>IR-Explain-1</literal> database;
- see <xref linkend="querymodel-exp1"/>.
+ see <xref linkend="querymodel-exp1"/>.
</para>
<para>
In &acro.sru;, explain is an entirely separate
- operation, which returns an ZeeRex &acro.xml; record according to the
+ operation, which returns an ZeeRex &acro.xml; record according to the
structure defined by the protocol.
</para>
<para>
In both cases, the information gathered through
explain operations can be used to
auto-configure a client user interface to the servers
- capabilities.
+ capabilities.
</para>
</section>
<section id="querymodel-operation-type-search">
<title>Search Operation</title>
<para>
- Search and retrieve interactions are the raison d'être.
+ Search and retrieve interactions are the raison d'être.
They are used to query the remote database and
return search result documents. Search queries span from
simple free text searches to nested complex boolean queries,
<title>Scan Operation</title>
<para>
The scan operation is a helper functionality,
- which operates on one index or access point a time.
+ which operates on one index or access point a time.
</para>
<para>
It provides
the indexes, and in addition the scan
operation returns the number of documents indexed by each term.
A search client can use this information to propose proper
- spelling of search terms, to auto-fill search boxes, or to
+ spelling of search terms, to auto-fill search boxes, or to
display controlled vocabularies.
</para>
</section>
</section>
- </section>
+ </section>
<section id="querymodel-rpn">
<title>&acro.rpn; queries and semantics</title>
is documented in the &yaz; manual, and shall not be
repeated here. This textual &acro.pqf; representation
is not transmitted to &zebra; during search, but it is in the
- client mapped to the equivalent &acro.z3950; binary
- query parse tree.
+ client mapped to the equivalent &acro.z3950; binary
+ query parse tree.
</para>
-
+
<section id="querymodel-rpn-tree">
<title>&acro.rpn; tree structure</title>
<para>
The &acro.rpn; parse tree - or the equivalent textual representation in &acro.pqf; -
- may start with one specification of the
+ may start with one specification of the
<emphasis>attribute set</emphasis> used. Following is a query
- tree, which
+ tree, which
consists of <emphasis>atomic query parts (&acro.apt;)</emphasis> or
<emphasis>named result sets</emphasis>, eventually
- paired by <emphasis>boolean binary operators</emphasis>, and
- finally <emphasis>recursively combined </emphasis> into
- complex query trees.
+ paired by <emphasis>boolean binary operators</emphasis>, and
+ finally <emphasis>recursively combined </emphasis> into
+ complex query trees.
</para>
-
+
<section id="querymodel-attribute-sets">
<title>Attribute sets</title>
<para>
definitions, others can easily be defined and added to the
configuration.
</para>
-
+
<table id="querymodel-attribute-sets-table" frame="top">
<title>Attribute sets predefined in &zebra;</title>
<tgroup cols="4">
<entry>Notes</entry>
</row>
</thead>
-
+
<tbody>
<row>
<entry>Explain</entry>
<entry><literal>bib-1</literal></entry>
<entry>Standard &acro.pqf; query language attribute set which defines the
semantics of &acro.z3950; searching. In addition, all of the
- non-use attributes (types 2-14) define the hard-wired
+ non-use attributes (types 2-14) define the hard-wired
&zebra; internal query
processing.</entry>
<entry>default</entry>
</tbody>
</tgroup>
</table>
-
+
<para>
The use attributes (type 1) mappings the
predefined attribute sets are found in the
- attribute set configuration files <filename>tab/*.att</filename>.
+ attribute set configuration files <filename>tab/*.att</filename>.
</para>
-
+
<note>
<para>
- The &zebra; internal query processing is modeled after
+ The &zebra; internal query processing is modeled after
the &acro.bib1; attribute set, and the non-use
attributes type 2-6 are hard-wired in. It is therefore essential
- to be familiar with <xref linkend="querymodel-bib1-nonuse"/>.
+ to be familiar with <xref linkend="querymodel-bib1-nonuse"/>.
</para>
</note>
-
+
</section>
-
+
<section id="querymodel-boolean-operators">
<title>Boolean operators</title>
<para>
using the standard boolean operators into new query trees.
Thus, boolean operators are always internal nodes in the query tree.
</para>
-
+
<table id="querymodel-boolean-operators-table" frame="top">
<title>Boolean operators</title>
<tgroup cols="3">
</row>
<row><entry><literal>@prox</literal></entry>
<entry>binary PROXIMITY operator</entry>
- <entry>Set intersection of two atomic queries hit sets. In
- addition, the intersection set is purged for all
- documents which do not satisfy the requested query
- term proximity. Usually a proper subset of the AND
+ <entry>Set intersection of two atomic queries hit sets. In
+ addition, the intersection set is purged for all
+ documents which do not satisfy the requested query
+ term proximity. Usually a proper subset of the AND
operation.</entry>
</row>
</tbody>
</tgroup>
</table>
-
+
<para>
- For example, we can combine the terms
- <emphasis>information</emphasis> and <emphasis>retrieval</emphasis>
+ For example, we can combine the terms
+ <emphasis>information</emphasis> and <emphasis>retrieval</emphasis>
into different searches in the default index of the default
attribute set as follows.
Querying for the union of all documents containing the
terms <emphasis>information</emphasis> OR
- <emphasis>retrieval</emphasis>:
+ <emphasis>retrieval</emphasis>:
<screen>
Z> find @or information retrieval
</screen>
<para>
Querying for the intersection of all documents containing the
terms <emphasis>information</emphasis> AND
- <emphasis>retrieval</emphasis>:
+ <emphasis>retrieval</emphasis>:
The hit set is a subset of the corresponding
OR query.
<screen>
terms <emphasis>information</emphasis> AND
<emphasis>retrieval</emphasis>, taking proximity into account:
The hit set is a subset of the corresponding
- AND query
+ AND query
(see the <ulink url="&url.yaz.pqf;">&acro.pqf; grammar</ulink> for
details on the proximity operator):
<screen>
Querying for the intersection of all documents containing the
terms <emphasis>information</emphasis> AND
<emphasis>retrieval</emphasis>, in the same order and near each
- other as described in the term list.
+ other as described in the term list.
The hit set is a subset of the corresponding
PROXIMITY query.
<screen>
</screen>
</para>
</section>
-
-
+
+
<section id="querymodel-atomic-queries">
<title>Atomic queries (&acro.apt;)</title>
<para>
Atomic queries are the query parts which work on one access point
only. These consist of <emphasis>an attribute list</emphasis>
followed by a <emphasis>single term</emphasis> or a
- <emphasis>quoted term list</emphasis>, and are often called
+ <emphasis>quoted term list</emphasis>, and are often called
<emphasis>Attributes-Plus-Terms (&acro.apt;)</emphasis> queries.
</para>
<para>
- Atomic (&acro.apt;) queries are always leaf nodes in the &acro.pqf; query tree.
+ Atomic (&acro.apt;) queries are always leaf nodes in the &acro.pqf; query tree.
UN-supplied non-use attributes types 2-12 are either inherited from
higher nodes in the query tree, or are set to &zebra;'s default values.
- See <xref linkend="querymodel-bib1"/> for details.
+ See <xref linkend="querymodel-bib1"/> for details.
</para>
-
+
<table id="querymodel-atomic-queries-table" frame="top">
<title>Atomic queries (&acro.apt;)</title>
<tgroup cols="3">
<entry>Type</entry>
<entry>Notes</entry>
</row>
- </thead>
+ </thead>
<tbody>
<row>
<entry><emphasis>attribute list</emphasis></entry>
</row>
<row>
<entry><emphasis>term</emphasis></entry>
- <entry>single <emphasis>term</emphasis>
+ <entry>single <emphasis>term</emphasis>
or <emphasis>quoted term list</emphasis> </entry>
<entry>Here the search terms or list of search terms is added
to the query</entry>
</para>
<para>
- The <emphasis>scan</emphasis> operation is only supported with
+ The <emphasis>scan</emphasis> operation is only supported with
atomic &acro.apt; queries, as it is bound to one access point at a
time. Boolean query trees are not allowed during
<emphasis>scan</emphasis>.
- </para>
-
+ </para>
+
<para>
For example, we might want to scan the title index, starting with
- the term
+ the term
<emphasis>debussy</emphasis>, and displaying this and the
following terms in lexicographic order:
<screen>
</screen>
</para>
</section>
-
-
+
+
<section id="querymodel-resultset">
<title>Named Result Sets</title>
<para>
result sets are leaf nodes in the &acro.pqf; query tree, exactly as
atomic &acro.apt; queries are.
</para>
- <para>
+ <para>
After the execution of a search, the result set is available at
the server, such that the client can use it for subsequent
searches or retrieval requests. The Z30.50 standard actually
send a diagnostic to the effect that the requested
result set does not exist any more.
</para>
-
+
<para>
Defining a named result set and re-using it in the next query,
using <application>yaz-client</application>. Notice that the client, not
the server, assigns the string '1' to the
- named result set.
+ named result set.
<screen>
Z> f @attr 1=4 mozart
...
Number of hits: 14, setno 2
</screen>
</para>
-
+
<note>
<para>
Named result sets are only supported by the &acro.z3950; protocol.
</para>
</note>
</section>
-
+
<section id="querymodel-use-string">
<title>&zebra;'s special access point of type 'string'</title>
<para>
- The numeric <emphasis>use (type 1)</emphasis> attribute is usually
+ The numeric <emphasis>use (type 1)</emphasis> attribute is usually
referred to from a given
- attribute set. In addition, &zebra; let you use
+ attribute set. In addition, &zebra; let you use
<emphasis>any internal index
- name defined in your configuration</emphasis>
+ name defined in your configuration</emphasis>
as use attribute value. This is a great feature for
debugging, and when you do
not need the complexity of defined use attribute values. It is
- the preferred way of accessing &zebra; indexes directly.
+ the preferred way of accessing &zebra; indexes directly.
</para>
<para>
Finding all documents which have the term list "information
<para>
It is possible to search
in any silly string index - if it's defined in your
- indexing rules and can be parsed by the &acro.pqf; parser.
+ indexing rules and can be parsed by the &acro.pqf; parser.
This is definitely not the recommended use of
this facility, as it might confuse your users with some very
unexpected results.
</screen>
</para>
<para>
- See also <xref linkend="querymodel-pqf-apt-mapping"/> for details, and
+ See also <xref linkend="querymodel-pqf-apt-mapping"/> for details, and
<xref linkend="zebrasrv-sru"/>
for the &acro.sru; &acro.pqf; query extension using string names as a fast
debugging facility.
</para>
</section>
-
+
<section id="querymodel-use-xpath">
- <title>&zebra;'s special access point of type 'XPath'
+ <title>&zebra;'s special access point of type 'XPath'
for &acro.grs1; filters</title>
<para>
As we have seen above, it is possible (albeit seldom a great
- idea) to emulate
+ idea) to emulate
<ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink> based
search by defining <emphasis>use (type 1)</emphasis>
- <emphasis>string</emphasis> attributes which in appearance
+ <emphasis>string</emphasis> attributes which in appearance
<emphasis>resemble XPath queries</emphasis>. There are two
problems with this approach: first, the XPath-look-alike has to
be defined at indexing time, no new undefined
XPath queries can entered at search time, and second, it might
confuse users very much that an XPath-alike index name in fact
gets populated from a possible entirely different &acro.xml; element
- than it pretends to access.
+ than it pretends to access.
</para>
<para>
When using the &acro.grs1; Record Model
(see <xref linkend="grs"/>), we have the
- possibility to embed <emphasis>life</emphasis>
+ possibility to embed <emphasis>life</emphasis>
XPath expressions
in the &acro.pqf; queries, which are here called
<emphasis>use (type 1)</emphasis> <emphasis>xpath</emphasis>
- attributes. You must enable the
- <literal>xpath enable</literal> directive in your
- <literal>.abs</literal> configuration files.
+ attributes. You must enable the
+ <literal>xpath enable</literal> directive in your
+ <literal>.abs</literal> configuration files.
</para>
<note>
<para>
- Only a <emphasis>very</emphasis> restricted subset of the
- <ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink>
+ Only a <emphasis>very</emphasis> restricted subset of the
+ <ulink url="http://www.w3.org/TR/xpath">XPath 1.0</ulink>
standard is supported as the &acro.grs1; record model is simpler than
- a full &acro.xml; &acro.dom; structure. See the following examples for
+ a full &acro.xml; &acro.dom; structure. See the following examples for
possibilities.
</para>
</note>
<para>
- Finding all documents which have the term "content"
+ Finding all documents which have the term "content"
inside a text node found in a specific &acro.xml; &acro.dom;
- <emphasis>subtree</emphasis>, whose starting element is
- addressed by XPath.
+ <emphasis>subtree</emphasis>, whose starting element is
+ addressed by XPath.
<screen>
- Z> find @attr 1=/root content
+ Z> find @attr 1=/root content
Z> find @attr 1=/root/first content
</screen>
<emphasis>Notice that the
</emphasis>
It follows that the above searches are interpreted as:
<screen>
- Z> find @attr 1=/root//text() content
+ Z> find @attr 1=/root//text() content
Z> find @attr 1=/root/first//text() content
</screen>
</para>
-
+
<para>
Searching inside attribute strings is possible:
<screen>
- Z> find @attr 1=/link/@creator morten
+ Z> find @attr 1=/link/@creator morten
</screen>
- </para>
+ </para>
- <para>
+ <para>
Filter the addressing XPath by a predicate working on exact
string values in
attributes (in the &acro.xml; sense) can be done: return all those docs which
<screen>
Z> find @attr 1=/record/title[@lang='en'] english
Z> find @attr 1=/link[@creator='sisse'] sibelius
- Z> find @attr 1=/link[@creator='sisse']/description[@xml:lang='da'] sibelius
+ Z> find @attr 1=/link[@creator='sisse']/description[@xml:lang='da'] sibelius
</screen>
</para>
-
- <para>
- Combining numeric indexes, boolean expressions,
+
+ <para>
+ Combining numeric indexes, boolean expressions,
and xpath based searches is possible:
<screen>
Z> find @attr 1=/record/title @and foo bar
Z> find @and @attr 1=/record/title foo @attr 1=4 bar
- </screen>
+ </screen>
</para>
<para>
Escaping &acro.pqf; keywords and other non-parseable XPath constructs
</warning>
</section>
</section>
-
+
<section id="querymodel-exp1">
<title>Explain Attribute Set</title>
<para>
- The &acro.z3950; standard defines the
+ The &acro.z3950; standard defines the
<ulink url="&url.z39.50.explain;">Explain</ulink> attribute set
- Exp-1, which is used to discover information
+ Exp-1, which is used to discover information
about a server's search semantics and functional capabilities
- &zebra; exposes a "classic"
+ &zebra; exposes a "classic"
Explain database by base name <literal>IR-Explain-1</literal>, which
- is populated with system internal information.
+ is populated with system internal information.
</para>
- <para>
- The attribute-set <literal>exp-1</literal> consists of a single
- use attribute (type 1).
+ <para>
+ The attribute-set <literal>exp-1</literal> consists of a single
+ use attribute (type 1).
</para>
<para>
In addition, the non-Use
- &acro.bib1; attributes, that is, the types
+ &acro.bib1; attributes, that is, the types
<emphasis>Relation</emphasis>, <emphasis>Position</emphasis>,
- <emphasis>Structure</emphasis>, <emphasis>Truncation</emphasis>,
- and <emphasis>Completeness</emphasis> are imported from
+ <emphasis>Structure</emphasis>, <emphasis>Truncation</emphasis>,
+ and <emphasis>Completeness</emphasis> are imported from
the &acro.bib1; attribute set, and may be used
- within any explain query.
+ within any explain query.
</para>
-
+
<section id="querymodel-exp1-use">
- <title>Use Attributes (type = 1)</title>
+ <title>Use Attributes (type = 1)</title>
<para>
The following Explain search attributes are supported:
- <literal>ExplainCategory</literal> (@attr 1=1),
- <literal>DatabaseName</literal> (@attr 1=3),
- <literal>DateAdded</literal> (@attr 1=9),
+ <literal>ExplainCategory</literal> (@attr 1=1),
+ <literal>DatabaseName</literal> (@attr 1=3),
+ <literal>DateAdded</literal> (@attr 1=9),
<literal>DateChanged</literal>(@attr 1=10).
</para>
<para>
<literal>DatabaseInfo</literal>, <literal>AttributeDetails</literal>.
</para>
<para>
- See <filename>tab/explain.att</filename> and the
+ See <filename>tab/explain.att</filename> and the
<ulink url="&url.z39.50;">&acro.z3950;</ulink> standard
for more information.
</para>
</section>
-
+
<section id="querymodel-examples">
<title>Explain searches with yaz-client</title>
<para>
via ASN.1. Practically no &acro.z3950; clients supports this. Fortunately
they don't have to - &zebra; allows retrieval of this information
in other formats:
- &acro.sutrs;, &acro.xml;,
+ &acro.sutrs;, &acro.xml;,
&acro.grs1; and <literal>ASN.1</literal> Explain.
</para>
-
+
<para>
List supported categories to find out which explain commands are
- supported:
+ supported:
<screen>
Z> base IR-Explain-1
Z> find @attr exp1 1=1 categorylist
Z> show 1+2
</screen>
</para>
-
+
<para>
Get target info, that is, investigate which databases exist at
this server endpoint:
Z> show 1+1
</screen>
</para>
-
+
<para>
List all supported databases, the number of hits
is the number of databases found, which most commonly are the
Z> show 1+2
</screen>
</para>
-
+
<para>
Get database info record for database <literal>Default</literal>.
<screen>
Z> find @attrset exp1 @and @attr 1=1 databaseinfo @attr 1=3 Default
</screen>
</para>
-
+
<para>
Get attribute details record for database
<literal>Default</literal>.
</screen>
</para>
</section>
-
+
</section>
-
+
<section id="querymodel-bib1">
<title>&acro.bib1; Attribute Set</title>
<para>
Most of the information contained in this section is an excerpt of
the ATTRIBUTE SET &acro.bib1; (&acro.z3950;-1995) SEMANTICS
found at <ulink url="&url.z39.50.attset.bib1.1995;">. The &acro.bib1;
- Attribute Set Semantics</ulink> from 1995, also in an updated
+ Attribute Set Semantics</ulink> from 1995, also in an updated
<ulink url="&url.z39.50.attset.bib1;">&acro.bib1;
- Attribute Set</ulink>
+ Attribute Set</ulink>
version from 2003. Index Data is not the copyright holder of this
information, except for the configuration details, the listing of
- &zebra;'s capabilities, and the example queries.
+ &zebra;'s capabilities, and the example queries.
</para>
-
-
- <section id="querymodel-bib1-use">
+
+
+ <section id="querymodel-bib1-use">
<title>Use Attributes (type 1)</title>
- <para>
- A use attribute specifies an access point for any atomic query.
- These access points are highly dependent on the attribute set used
- in the query, and are user configurable using the following
- default configuration files:
- <filename>tab/bib1.att</filename>,
- <filename>tab/dan1.att</filename>,
- <filename>tab/explain.att</filename>, and
- <filename>tab/gils.att</filename>.
+ <para>
+ A use attribute specifies an access point for any atomic query.
+ These access points are highly dependent on the attribute set used
+ in the query, and are user configurable using the following
+ default configuration files:
+ <filename>tab/bib1.att</filename>,
+ <filename>tab/dan1.att</filename>,
+ <filename>tab/explain.att</filename>, and
+ <filename>tab/gils.att</filename>.
</para>
- <para>
+ <para>
For example, some few &acro.bib1; use
attributes from the <filename>tab/bib1.att</filename> are:
<screen>
att 1036 Author-Title-Subject
</screen>
</para>
- <para>
- New attribute sets can be added by adding new
- <filename>tab/*.att</filename> configuration files, which need to
- be sourced in the main configuration <filename>zebra.cfg</filename>.
+ <para>
+ New attribute sets can be added by adding new
+ <filename>tab/*.att</filename> configuration files, which need to
+ be sourced in the main configuration <filename>zebra.cfg</filename>.
+ </para>
+ <para>
+ In addition, &zebra; allows the access of
+ <emphasis>internal index names</emphasis> and <emphasis>dynamic
+ XPath</emphasis> as use attributes; see
+ <xref linkend="querymodel-use-string"/> and
+ <xref linkend="querymodel-use-xpath"/>.
</para>
- <para>
- In addition, &zebra; allows the access of
- <emphasis>internal index names</emphasis> and <emphasis>dynamic
- XPath</emphasis> as use attributes; see
- <xref linkend="querymodel-use-string"/> and
- <xref linkend="querymodel-use-xpath"/>.
- </para>
- <para>
- Phrase search for <emphasis>information retrieval</emphasis> in
- the title-register, scanning the same register afterwards:
- <screen>
- Z> find @attr 1=4 "information retrieval"
- Z> scan @attr 1=4 information
- </screen>
- </para>
+ <para>
+ Phrase search for <emphasis>information retrieval</emphasis> in
+ the title-register, scanning the same register afterwards:
+ <screen>
+ Z> find @attr 1=4 "information retrieval"
+ Z> scan @attr 1=4 information
+ </screen>
+ </para>
</section>
</section>
<section id="querymodel-bib1-nonuse">
- <title>&zebra; general Bib1 Non-Use Attributes (type 2-6)</title>
+ <title>&zebra; general Bib1 Non-Use Attributes (type 2-6)</title>
<section id="querymodel-bib1-relation">
<title>Relation Attributes (type 2)</title>
-
+
<para>
Relation attributes describe the relationship of the access
- point (left side
+ point (left side
of the relation) to the search term as qualified by the attributes (right
side of the relation), e.g., Date-publication <= 1975.
- </para>
+ </para>
<table id="querymodel-bib1-relation-table" frame="top">
<title>Relation Attributes (type 2)</title>
AlwaysMatches searches are only supported if alwaysmatches indexing
has been enabled. See <xref linkend="default-idx-file"/>
</para>
- </note>
-
+ </note>
+
<para>
The relation attributes 1-5 are supported and work exactly as
expected.
- All ordering operations are based on a lexicographical ordering,
- <emphasis>except</emphasis> when the
+ All ordering operations are based on a lexicographical ordering,
+ <emphasis>except</emphasis> when the
structure attribute numeric (109) is used. In
- this case, ordering is numerical. See
+ this case, ordering is numerical. See
<xref linkend="querymodel-bib1-structure"/>.
<screen>
Z> find @attr 1=Title @attr 2=1 music
</para>
<para>
- The relation attribute
+ The relation attribute
<emphasis>Relevance (102)</emphasis> is supported, see
<xref linkend="administration-ranking"/> for full information.
</para>
-
+
<para>
Ranked search for <emphasis>information retrieval</emphasis> in
the title-register:
</para>
<para>
- The relation attribute
+ The relation attribute
<emphasis>AlwaysMatches (103)</emphasis> is in the default
configuration
- supported in conjecture with structure attribute
+ supported in conjecture with structure attribute
<emphasis>Phrase (1)</emphasis> (which may be omitted by
- default).
+ default).
It can be configured to work with other structure attributes,
- see the configuration file
- <filename>tab/default.idx</filename> and
- <xref linkend="querymodel-pqf-apt-mapping"/>.
+ see the configuration file
+ <filename>tab/default.idx</filename> and
+ <xref linkend="querymodel-pqf-apt-mapping"/>.
</para>
<para>
<emphasis>AlwaysMatches (103)</emphasis> is a
<section id="querymodel-bib1-position">
<title>Position Attributes (type 3)</title>
-
+
<para>
The position attribute specifies the location of the search term
within the field or subfield in which it appears.
<section id="querymodel-bib1-structure">
<title>Structure Attributes (type 4)</title>
-
+
<para>
The structure attribute specifies the type of search
term. This causes the search to be mapped on
different &zebra; internal indexes, which must have been defined
- at index time.
+ at index time.
</para>
- <para>
- The possible values of the
+ <para>
+ The possible values of the
<literal>structure attribute (type 4)</literal> can be defined
using the configuration file <filename>tab/default.idx</filename>.
The default configuration is summarized in this table.
</tbody>
</tgroup>
</table>
- <para>
- The structure attribute values
- <literal>Word list (6)</literal>
- is supported, and maps to the boolean <literal>AND</literal>
- combination of words supplied. The word list is useful when
- Google-like bag-of-word queries need to be translated from a GUI
- query language to &acro.pqf;. For example, the following queries
- are equivalent:
- <screen>
- Z> find @attr 1=Title @attr 4=6 "mozart amadeus"
- Z> find @attr 1=Title @and mozart amadeus
- </screen>
- </para>
+ <para>
+ The structure attribute values
+ <literal>Word list (6)</literal>
+ is supported, and maps to the boolean <literal>AND</literal>
+ combination of words supplied. The word list is useful when
+ Google-like bag-of-word queries need to be translated from a GUI
+ query language to &acro.pqf;. For example, the following queries
+ are equivalent:
+ <screen>
+ Z> find @attr 1=Title @attr 4=6 "mozart amadeus"
+ Z> find @attr 1=Title @and mozart amadeus
+ </screen>
+ </para>
- <para>
- The structure attribute value
- <literal>Free-form-text (105)</literal> and
- <literal>Document-text (106)</literal>
- are supported, and map both to the boolean <literal>OR</literal>
- combination of words supplied. The following queries
- are equivalent:
- <screen>
- Z> find @attr 1=Body-of-text @attr 4=105 "bach salieri teleman"
- Z> find @attr 1=Body-of-text @attr 4=106 "bach salieri teleman"
- Z> find @attr 1=Body-of-text @or bach @or salieri teleman
- </screen>
- This <literal>OR</literal> list of terms is very useful in
- combination with relevance ranking:
- <screen>
- Z> find @attr 1=Body-of-text @attr 2=102 @attr 4=105 "bach salieri teleman"
- </screen>
- </para>
+ <para>
+ The structure attribute value
+ <literal>Free-form-text (105)</literal> and
+ <literal>Document-text (106)</literal>
+ are supported, and map both to the boolean <literal>OR</literal>
+ combination of words supplied. The following queries
+ are equivalent:
+ <screen>
+ Z> find @attr 1=Body-of-text @attr 4=105 "bach salieri teleman"
+ Z> find @attr 1=Body-of-text @attr 4=106 "bach salieri teleman"
+ Z> find @attr 1=Body-of-text @or bach @or salieri teleman
+ </screen>
+ This <literal>OR</literal> list of terms is very useful in
+ combination with relevance ranking:
+ <screen>
+ Z> find @attr 1=Body-of-text @attr 2=102 @attr 4=105 "bach salieri teleman"
+ </screen>
+ </para>
- <para>
- The structure attribute value
- <literal>Local number (107)</literal>
- is supported, and maps always to the &zebra; internal document ID,
- irrespectively which use attribute is specified. The following queries
- have exactly the same unique record in the hit set:
- <screen>
- Z> find @attr 4=107 10
- Z> find @attr 1=4 @attr 4=107 10
- Z> find @attr 1=1010 @attr 4=107 10
- </screen>
- </para>
+ <para>
+ The structure attribute value
+ <literal>Local number (107)</literal>
+ is supported, and maps always to the &zebra; internal document ID,
+ irrespectively which use attribute is specified. The following queries
+ have exactly the same unique record in the hit set:
+ <screen>
+ Z> find @attr 4=107 10
+ Z> find @attr 1=4 @attr 4=107 10
+ Z> find @attr 1=1010 @attr 4=107 10
+ </screen>
+ </para>
- <para>
- In
- the GILS schema (<literal>gils.abs</literal>), the
- west-bounding-coordinate is indexed as type <literal>n</literal>,
- and is therefore searched by specifying
- <emphasis>structure</emphasis>=<emphasis>Numeric String</emphasis>.
- To match all those records with west-bounding-coordinate greater
- than -114 we use the following query:
- <screen>
- Z> find @attr 4=109 @attr 2=5 @attr gils 1=2038 -114
- </screen>
+ <para>
+ In
+ the GILS schema (<literal>gils.abs</literal>), the
+ west-bounding-coordinate is indexed as type <literal>n</literal>,
+ and is therefore searched by specifying
+ <emphasis>structure</emphasis>=<emphasis>Numeric String</emphasis>.
+ To match all those records with west-bounding-coordinate greater
+ than -114 we use the following query:
+ <screen>
+ Z> find @attr 4=109 @attr 2=5 @attr gils 1=2038 -114
+ </screen>
</para>
<note>
<para>
The exact mapping between &acro.pqf; queries and &zebra; internal indexes
- and index types is explained in
+ and index types is explained in
<xref linkend="querymodel-pqf-apt-mapping"/>.
</para>
</note>
</section>
-
+
<section id="querymodel-bib1-truncation">
<title>Truncation Attributes (type = 5)</title>
...
Number of hits: 95, setno 8
</screen>
- </para>
+ </para>
<para>
- The truncation attribute value
+ The truncation attribute value
<literal>Process # in search term (101)</literal> is a
poor-man's regular expression search. It maps
each <literal>#</literal> to <literal>.*</literal>, and
Number of hits: 89, setno 10
</screen>
</para>
-
+
<para>
- The truncation attribute value
- <literal>Regexp-1 (102)</literal> is a normal regular search,
+ The truncation attribute value
+ <literal>Regexp-1 (102)</literal> is a normal regular search,
see <xref linkend="querymodel-regular"/> for details.
<screen>
Z> find @attr 1=Body-of-text @attr 5=102 schnit+ke
</para>
<para>
- The truncation attribute value
+ The truncation attribute value
<literal>Regexp-2 (103) </literal> is a &zebra; specific extension
which allows <emphasis>fuzzy</emphasis> matches. One single
error in spelling of search terms is allowed, i.e., a document
is hit if it includes a term which can be mapped to the used
search term by one character substitution, addition, deletion or
- change of position.
+ change of position.
<screen>
Z> find @attr 1=Body-of-text @attr 5=100 schnittke
...
Number of hits: 103, setno 15
...
</screen>
- </para>
+ </para>
</section>
-
+
<section id="querymodel-bib1-completeness">
- <title>Completeness Attributes (type = 6)</title>
+ <title>Completeness Attributes (type = 6)</title>
<para>
The <literal>Completeness Attributes (type = 6)</literal>
- is used to specify that a given search term or term list is either
- part of the terms of a given index/field
+ is used to specify that a given search term or term list is either
+ part of the terms of a given index/field
(<literal>Incomplete subfield (1)</literal>), or is
what literally is found in the entire field's index
(<literal>Complete field (3)</literal>).
- </para>
+ </para>
<table id="querymodel-bib1-completeness-table" frame="top">
<title>Completeness Attributes (type = 6)</title>
The <literal>Completeness Attributes (type = 6)</literal>
is only partially and conditionally
supported in the sense that it is ignored if the hit index is
- not of structure <literal>type="w"</literal> or
+ not of structure <literal>type="w"</literal> or
<literal>type="p"</literal>.
- </para>
+ </para>
<para>
<literal>Incomplete subfield (1)</literal> is the default, and
- makes &zebra; use
- register <literal>type="w"</literal>, whereas
+ makes &zebra; use
+ register <literal>type="w"</literal>, whereas
<literal>Complete field (3)</literal> triggers
search and scan in index <literal>type="p"</literal>.
</para>
<note>
<para>
The exact mapping between &acro.pqf; queries and &zebra; internal indexes
- and index types is explained in
+ and index types is explained in
<xref linkend="querymodel-pqf-apt-mapping"/>.
</para>
</note>
</section>
- </section>
+ </section>
<section id="querymodel-zebra">
<title>Extended &zebra; &acro.rpn; Features</title>
and <emphasis>non-portable</emphasis>: most functional extensions
are modeled over the <literal>bib-1</literal> attribute set,
defining type 7 and higher values.
- There are also the special
+ There are also the special
<literal>string</literal> type index names for the
- <literal>idxpath</literal> attribute set.
+ <literal>idxpath</literal> attribute set.
</para>
-
+
<section id="querymodel-zebra-attr-allrecords">
<title>&zebra; specific retrieval of all records</title>
<para>
&zebra; defines a hardwired <literal>string</literal> index name
called <literal>_ALLRECORDS</literal>. It matches any record
- contained in the database, if used in conjunction with
- the relation attribute
+ contained in the database, if used in conjunction with
+ the relation attribute
<literal>AlwaysMatches (103)</literal>.
- </para>
+ </para>
<para>
The <literal>_ALLRECORDS</literal> index name is used for total database
export. The search term is ignored, it may be empty.
<title>&zebra; specific Search Extensions to all Attribute Sets</title>
<para>
&zebra; extends the &acro.bib1; attribute types, and these extensions are
- recognized regardless of attribute
+ recognized regardless of attribute
set used in a <literal>search</literal> operation query.
</para>
-
+
<table id="querymodel-zebra-attr-search-table" frame="top">
<title>&zebra; Search Attribute Extensions</title>
<tgroup cols="4">
<thead>
<row>
- <entry>Name</entry>
+ <entry>Name</entry>
<entry>Value</entry>
<entry>Operation</entry>
<entry>&zebra; version</entry>
</row>
</tbody>
</tgroup>
- </table>
-
+ </table>
+
<section id="querymodel-zebra-attr-sorting">
<title>&zebra; Extension Embedded Sort Attribute (type 7)</title>
<para>
The embedded sort is a way to specify sort within a query - thus
removing the need to send a Sort Request separately. It is both
faster and does not require clients to deal with the Sort
- Facility.
+ Facility.
</para>
-
+
<para>
- All ordering operations are based on a lexicographical ordering,
- <emphasis>except</emphasis> when the
+ All ordering operations are based on a lexicographical ordering,
+ <emphasis>except</emphasis> when the
<literal>structure attribute numeric (109)</literal> is used. In
- this case, ordering is numerical. See
+ this case, ordering is numerical. See
<xref linkend="querymodel-bib1-structure"/>.
</para>
-
+
<para>
The possible values after attribute <literal>type 7</literal> are
- <literal>1</literal> ascending and
- <literal>2</literal> descending.
+ <literal>1</literal> ascending and
+ <literal>2</literal> descending.
The attributes+term (&acro.apt;) node is separate from the
- rest and must be <literal>@or</literal>'ed.
+ rest and must be <literal>@or</literal>'ed.
The term associated with &acro.apt; is the sorting level in integers,
- where <literal>0</literal> means primary sort,
+ where <literal>0</literal> means primary sort,
<literal>1</literal> means secondary sort, and so forth.
See also <xref linkend="administration-ranking"/>.
</para>
<para>
- For example, searching for water, sort by title (ascending)
+ For example, searching for water, sort by title (ascending)
<screen>
Z> find @or @attr 1=1016 water @attr 7=1 @attr 1=4 0
</screen>
</para>
</section>
- <!--
+ <!--
&zebra; Extension Term Set Attribute
From the manual text, I can not see what is the point with this feature.
I think it makes more sense when there are multiple terms in a query, or
something...
-
+
We decided 2006-06-03 to disable this feature, as it is covered by
scan within a resultset. Better use ressources to upgrade this
feature for good performance.
-->
- <!--
+ <!--
<section id="querymodel-zebra-attr-estimation">
- <title>&zebra; Extension Term Set Attribute (type 8)</title>
+ <title>&zebra; Extension Term Set Attribute (type 8)</title>
<para>
- The Term Set feature is a facility that allows a search to store
- hitting terms in a "pseudo" resultset; thus a search (as usual) +
- a scan-like facility. Requires a client that can do named result
- sets since the search generates two result sets. The value for
- attribute 8 is the name of a result set (string). The terms in
- the named term set are returned as &acro.sutrs; records.
- </para>
+ The Term Set feature is a facility that allows a search to store
+ hitting terms in a "pseudo" resultset; thus a search (as usual) +
+ a scan-like facility. Requires a client that can do named result
+ sets since the search generates two result sets. The value for
+ attribute 8 is the name of a result set (string). The terms in
+ the named term set are returned as &acro.sutrs; records.
+ </para>
<para>
- For example, searching for u in title, right truncated, and
- storing the result in term set named 'aset'
- <screen>
- Z> find @attr 5=1 @attr 1=4 @attr 8=aset u
- </screen>
- </para>
+ For example, searching for u in title, right truncated, and
+ storing the result in term set named 'aset'
+ <screen>
+ Z> find @attr 5=1 @attr 1=4 @attr 8=aset u
+ </screen>
+ </para>
<warning>
- The model has one serious flaw: we don't know the size of term
- set. Experimental. Do not use in production code.
- </warning>
- </section>
+ The model has one serious flaw: we don't know the size of term
+ set. Experimental. Do not use in production code.
+ </warning>
+ </section>
-->
<title>&zebra; Extension Rank Weight Attribute (type 9)</title>
<para>
Rank weight is a way to pass a value to a ranking algorithm - so
- that one &acro.apt; has one value - while another as a different one.
+ that one &acro.apt; has one value - while another as a different one.
See also <xref linkend="administration-ranking"/>.
</para>
<para>
For example, searching for utah in title with weight 30 as well
- as any with weight 20:
- <screen>
+ as any with weight 20:
+ <screen>
Z> find @attr 2=102 @or @attr 9=30 @attr 1=4 utah @attr 9=20 utah
</screen>
</para>
<section id="querymodel-zebra-attr-termref">
<title>&zebra; Extension Term Reference Attribute (type 10)</title>
<para>
- &zebra; supports the searchResult-1 facility.
+ &zebra; supports the searchResult-1 facility.
If the Term Reference Attribute (type 10) is
given, that specifies a subqueryId value returned as part of the
search result. It is a way for a client to name an &acro.apt; part of a
- query.
+ query.
</para>
<warning>
<para>
Experimental. Do not use in production code.
- </para>
+ </para>
</warning>
-
+
</section>
-
-
-
+
+
+
<section id="querymodel-zebra-local-attr-limit">
<title>Local Approximative Limit Attribute (type 11)</title>
<para>
</para>
<para>
For example, we might be interested in exact hit count for a, but
- for b we allow hit count estimates for 1000 and higher.
+ for b we allow hit count estimates for 1000 and higher.
<screen>
Z> find @and a @attr 11=1000 b
</screen>
The estimated hit count facility makes searches faster, as one
only needs to process large hit lists partially.
It is mostly used in huge databases, where you you want trade
- exactness of hit counts against speed of execution.
+ exactness of hit counts against speed of execution.
</para>
</note>
<warning>
Do not use approximative hit count limits
in conjunction with relevance ranking, as re-sorting of the
result set only works when the entire result set has
- been processed.
+ been processed.
</para>
</warning>
</section>
<para>
By default &zebra; computes precise hit counts for a query as
a whole. Setting attribute 12 makes it perform approximative
- hit counts instead. It has the same semantics as
+ hit counts instead. It has the same semantics as
<literal>estimatehits</literal> for the <xref linkend="zebra-cfg"/>.
</para>
<para>
Do not use approximative hit count limits
in conjunction with relevance ranking, as re-sorting of the
result set only works when the entire result set has
- been processed.
+ been processed.
</para>
</warning>
</section>
<title>&zebra; specific Scan Extensions to all Attribute Sets</title>
<para>
&zebra; extends the Bib1 attribute types, and these extensions are
- recognized regardless of attribute
+ recognized regardless of attribute
set used in a scan operation query.
</para>
<table id="querymodel-zebra-attr-scan-table" frame="top">
</row>
</tbody>
</tgroup>
- </table>
-
+ </table>
+
<section id="querymodel-zebra-attr-narrow">
<title>&zebra; Extension Result Set Narrow (type 8)</title>
<para>
If attribute Result Set Narrow (type 8)
is given for scan, the value is the name of a
- result set. Each hit count in scan is
- <literal>@and</literal>'ed with the result set given.
+ result set. Each hit count in scan is
+ <literal>@and</literal>'ed with the result set given.
</para>
<para>
- Consider for example
+ Consider for example
the case of scanning all title fields around the
scanterm <emphasis>mozart</emphasis>, then refining the scan by
issuing a filtering query for <emphasis>amadeus</emphasis> to
- restrict the scan to the result set of the query:
+ restrict the scan to the result set of the query:
<screen>
- Z> scan @attr 1=4 mozart
- ...
- * mozart (43)
- mozartforskningen (1)
- mozartiana (1)
- mozarts (16)
- ...
- Z> f @attr 1=4 amadeus
- ...
- Number of hits: 15, setno 2
- ...
- Z> scan @attr 1=4 @attr 8=2 mozart
- ...
- * mozart (14)
- mozartforskningen (0)
- mozartiana (0)
- mozarts (1)
- ...
+ Z> scan @attr 1=4 mozart
+ ...
+ * mozart (43)
+ mozartforskningen (1)
+ mozartiana (1)
+ mozarts (16)
+ ...
+ Z> f @attr 1=4 amadeus
+ ...
+ Number of hits: 15, setno 2
+ ...
+ Z> scan @attr 1=4 @attr 8=2 mozart
+ ...
+ * mozart (14)
+ mozartforskningen (0)
+ mozartiana (0)
+ mozarts (1)
+ ...
</screen>
</para>
-
+
<para>
&zebra; 2.0.2 and later is able to skip 0 hit counts. This, however,
is known not to scale if the number of terms to skip is high.
<para>
The &zebra; Extension Approximative Limit (type 12) is a way to
enable approximate hit counts for scan hit counts, in the same
- way as for search hit counts.
+ way as for search hit counts.
</para>
</section>
</section>
-
+
<section id="querymodel-idxpath">
<title>&zebra; special &acro.idxpath; Attribute Set for &acro.grs1; indexing</title>
<para>
- The attribute-set <literal>idxpath</literal> consists of a single
- Use (type 1) attribute. All non-use attributes behave as normal.
+ The attribute-set <literal>idxpath</literal> consists of a single
+ Use (type 1) attribute. All non-use attributes behave as normal.
</para>
<para>
This feature is enabled when defining the
</warning>
<section id="querymodel-idxpath-use">
- <title>&acro.idxpath; Use Attributes (type = 1)</title>
+ <title>&acro.idxpath; Use Attributes (type = 1)</title>
<para>
This attribute set allows one to search &acro.grs1; filter indexed
- records by &acro.xpath; like structured index names.
+ records by &acro.xpath; like structured index names.
</para>
<warning>
index names, which might clash with your own index names.
</para>
</warning>
-
+
<table id="querymodel-idxpath-use-table" frame="top">
<title>&zebra; specific &acro.idxpath; Use Attributes (type 1)</title>
<tgroup cols="4">
See <filename>tab/idxpath.att</filename> for more information.
</para>
<para>
- Search for all documents starting with root element
+ Search for all documents starting with root element
<literal>/root</literal> (either using the numeric or the string
use attributes):
<screen>
- Z> find @attrset idxpath @attr 1=1 @attr 4=3 root/
- Z> find @attr idxpath 1=1 @attr 4=3 root/
- Z> find @attr 1=_XPATH_BEGIN @attr 4=3 root/
+ Z> find @attrset idxpath @attr 1=1 @attr 4=3 root/
+ Z> find @attr idxpath 1=1 @attr 4=3 root/
+ Z> find @attr 1=_XPATH_BEGIN @attr 4=3 root/
</screen>
</para>
<para>
- Search for all documents where specific nested &acro.xpath;
+ Search for all documents where specific nested &acro.xpath;
<literal>/c1/c2/../cn</literal> exists. Notice the very
counter-intuitive <emphasis>reverse</emphasis> notation!
<screen>
- Z> find @attrset idxpath @attr 1=1 @attr 4=3 cn/cn-1/../c1/
- Z> find @attr 1=_XPATH_BEGIN @attr 4=3 cn/cn-1/../c1/
+ Z> find @attrset idxpath @attr 1=1 @attr 4=3 cn/cn-1/../c1/
+ Z> find @attr 1=_XPATH_BEGIN @attr 4=3 cn/cn-1/../c1/
</screen>
</para>
<para>
</screen>
</para>
<para>
- Search for CDATA string <emphasis>anothertext</emphasis> in any
- attribute:
- <screen>
+ Search for CDATA string <emphasis>anothertext</emphasis> in any
+ attribute:
+ <screen>
Z> find @attrset idxpath @attr 1=1015 anothertext
Z> find @attr 1=_XPATH_ATTR_CDATA anothertext
</screen>
</para>
<para>
- Search for all documents with have an &acro.xml; element node
- including an &acro.xml; attribute named <emphasis>creator</emphasis>
- <screen>
- Z> find @attrset idxpath @attr 1=3 @attr 4=3 creator
- Z> find @attr 1=_XPATH_ATTR_NAME @attr 4=3 creator
+ Search for all documents with have an &acro.xml; element node
+ including an &acro.xml; attribute named <emphasis>creator</emphasis>
+ <screen>
+ Z> find @attrset idxpath @attr 1=3 @attr 4=3 creator
+ Z> find @attr 1=_XPATH_ATTR_NAME @attr 4=3 creator
</screen>
</para>
<para>
<para>
Scanning is supported on all <literal>idxpath</literal>
indexes, both specified as numeric use attributes, or as string
- index names.
+ index names.
<screen>
Z> scan @attrset idxpath @attr 1=1016 text
Z> scan @attr 1=_XPATH_ATTR_CDATA anothertext
<section id="querymodel-pqf-apt-mapping">
- <title>Mapping from &acro.pqf; atomic &acro.apt; queries to &zebra; internal
+ <title>Mapping from &acro.pqf; atomic &acro.apt; queries to &zebra; internal
register indexes</title>
<para>
The rules for &acro.pqf; &acro.apt; mapping are rather tricky to grasp in the
internal register or string index to use, according to the use
attribute or access point specified in the query. Thereafter we
deal with the rules for determining the correct structure type of
- the named register.
+ the named register.
</para>
- <section id="querymodel-pqf-apt-mapping-accesspoint">
- <title>Mapping of &acro.pqf; &acro.apt; access points</title>
- <para>
+ <section id="querymodel-pqf-apt-mapping-accesspoint">
+ <title>Mapping of &acro.pqf; &acro.apt; access points</title>
+ <para>
&zebra; understands four fundamental different types of access
- points, of which only the
+ points, of which only the
<emphasis>numeric use attribute</emphasis> type access points
are defined by the <ulink url="&url.z39.50;">&acro.z3950;</ulink>
standard.
All other access point types are &zebra; specific, and non-portable.
- </para>
+ </para>
<table id="querymodel-zebra-mapping-accesspoint-types" frame="top">
<title>Access point name mapping</title>
<entry>Grammar</entry>
<entry>Notes</entry>
</row>
- </thead>
- <tbody>
- <row>
- <entry>Use attribute</entry>
- <entry>numeric</entry>
- <entry>[1-9][1-9]*</entry>
- <entry>directly mapped to string index name</entry>
- </row>
- <row>
- <entry>String index name</entry>
- <entry>string</entry>
- <entry>[a-zA-Z](\-?[a-zA-Z0-9])*</entry>
- <entry>normalized name is used as internal string index name</entry>
- </row>
- <row>
- <entry>&zebra; internal index name</entry>
- <entry>zebra</entry>
- <entry>_[a-zA-Z](_?[a-zA-Z0-9])*</entry>
- <entry>hardwired internal string index name</entry>
- </row>
- <row>
- <entry>&acro.xpath; special index</entry>
- <entry>XPath</entry>
- <entry>/.*</entry>
- <entry>special xpath search for &acro.grs1; indexed records</entry>
- </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>Use attribute</entry>
+ <entry>numeric</entry>
+ <entry>[1-9][1-9]*</entry>
+ <entry>directly mapped to string index name</entry>
+ </row>
+ <row>
+ <entry>String index name</entry>
+ <entry>string</entry>
+ <entry>[a-zA-Z](\-?[a-zA-Z0-9])*</entry>
+ <entry>normalized name is used as internal string index name</entry>
+ </row>
+ <row>
+ <entry>&zebra; internal index name</entry>
+ <entry>zebra</entry>
+ <entry>_[a-zA-Z](_?[a-zA-Z0-9])*</entry>
+ <entry>hardwired internal string index name</entry>
+ </row>
+ <row>
+ <entry>&acro.xpath; special index</entry>
+ <entry>XPath</entry>
+ <entry>/.*</entry>
+ <entry>special xpath search for &acro.grs1; indexed records</entry>
+ </row>
</tbody>
</tgroup>
</table>
-
+
<para>
- <literal>Attribute set names</literal> and
+ <literal>Attribute set names</literal> and
<literal>string index names</literal> are normalizes
according to the following rules: all <emphasis>single</emphasis>
hyphens <literal>'-'</literal> are stripped, and all upper case
letters are folded to lower case.
</para>
-
+
<para>
- <emphasis>Numeric use attributes</emphasis> are mapped
+ <emphasis>Numeric use attributes</emphasis> are mapped
to the &zebra; internal
string index according to the attribute set definition in use.
The default attribute set is &acro.bib1;, and may be
omitted in the &acro.pqf; query.
</para>
-
+
<para>
According to normalization and numeric
use attribute mapping, it follows that the following
&acro.pqf; queries are considered equivalent (assuming the default
configuration has not been altered):
<screen>
- Z> find @attr 1=Body-of-text serenade
- Z> find @attr 1=bodyoftext serenade
- Z> find @attr 1=BodyOfText serenade
- Z> find @attr 1=bO-d-Y-of-tE-x-t serenade
- Z> find @attr 1=1010 serenade
- Z> find @attrset bib1 @attr 1=1010 serenade
- Z> find @attrset bib1 @attr 1=1010 serenade
- Z> find @attrset Bib1 @attr 1=1010 serenade
- Z> find @attrset b-I-b-1 @attr 1=1010 serenade
- </screen>
- </para>
+ Z> find @attr 1=Body-of-text serenade
+ Z> find @attr 1=bodyoftext serenade
+ Z> find @attr 1=BodyOfText serenade
+ Z> find @attr 1=bO-d-Y-of-tE-x-t serenade
+ Z> find @attr 1=1010 serenade
+ Z> find @attrset bib1 @attr 1=1010 serenade
+ Z> find @attrset bib1 @attr 1=1010 serenade
+ Z> find @attrset Bib1 @attr 1=1010 serenade
+ Z> find @attrset b-I-b-1 @attr 1=1010 serenade
+ </screen>
+ </para>
- <para>
+ <para>
The <emphasis>numerical</emphasis>
- <literal>use attributes (type 1)</literal>
+ <literal>use attributes (type 1)</literal>
are interpreted according to the
attribute sets which have been loaded in the
<literal>zebra.cfg</literal> file, and are matched against specific
fields as specified in the <literal>.abs</literal> file which
describes the profile of the records which have been loaded.
- If no use attribute is provided, a default of
+ If no use attribute is provided, a default of
&acro.bib1; Use Any (1016) is assumed.
The predefined use attribute sets
can be reconfigured by tweaking the configuration files
- <filename>tab/*.att</filename>, and
+ <filename>tab/*.att</filename>, and
new attribute sets can be defined by adding similar files in the
- configuration path <literal>profilePath</literal> of the server.
- </para>
+ configuration path <literal>profilePath</literal> of the server.
+ </para>
<para>
String indexes can be accessed directly,
<para>
&zebra; internal indexes can be accessed directly,
according to the same rules as the user defined
- string indexes. The only difference is that
+ string indexes. The only difference is that
&zebra; internal index names are hardwired,
all uppercase and
- must start with the character <literal>'_'</literal>.
+ must start with the character <literal>'_'</literal>.
</para>
<para>
available using the &acro.grs1; filter for indexing.
These access point names must start with the character
<literal>'/'</literal>, they are <emphasis>not
- normalized</emphasis>, but passed unaltered to the &zebra; internal
+ normalized</emphasis>, but passed unaltered to the &zebra; internal
&acro.xpath; engine. See <xref linkend="querymodel-use-xpath"/>.
</para>
</section>
- <section id="querymodel-pqf-apt-mapping-structuretype">
- <title>Mapping of &acro.pqf; &acro.apt; structure and completeness to
+ <section id="querymodel-pqf-apt-mapping-structuretype">
+ <title>Mapping of &acro.pqf; &acro.apt; structure and completeness to
register type</title>
- <para>
+ <para>
Internally &zebra; has in its default configuration several
- different types of registers or indexes, whose tokenization and
- character normalization rules differ. This reflects the fact that
+ different types of registers or indexes, whose tokenization and
+ character normalization rules differ. This reflects the fact that
searching fundamental different tokens like dates, numbers,
- bitfields and string based text needs different rule sets.
+ bitfields and string based text needs different rule sets.
</para>
<table id="querymodel-zebra-mapping-structure-types" frame="top">
<tbody>
<row>
<entry>
- phrase (@attr 4=1), word (@attr 4=2),
+ phrase (@attr 4=1), word (@attr 4=2),
word-list (@attr 4=6),
free-form-text (@attr 4=105), or document-text (@attr 4=106)
</entry>
</row>
<row>
<entry>
- phrase (@attr 4=1), word (@attr 4=2),
+ phrase (@attr 4=1), word (@attr 4=2),
word-list (@attr 4=6),
free-form-text (@attr 4=105), or document-text (@attr 4=106)
</entry>
<entry>overruled</entry>
<entry>overruled</entry>
<entry>special</entry>
- <entry>Internal record ID register, used whenever
+ <entry>Internal record ID register, used whenever
Relation Always Matches (@attr 2=103) is specified</entry>
</row>
</tbody>
</tgroup>
</table>
-
+
<!-- see in util/zebramap.c -->
-
- <para>
- If a <emphasis>Structure</emphasis> attribute of
- <emphasis>Phrase</emphasis> is used in conjunction with a
- <emphasis>Completeness</emphasis> attribute of
- <emphasis>Complete (Sub)field</emphasis>, the term is matched
- against the contents of the phrase (long word) register, if one
- exists for the given <emphasis>Use</emphasis> attribute.
- A phrase register is created for those fields in the
- &acro.grs1; <filename>*.abs</filename> file that contains a
- <literal>p</literal>-specifier.
+
+ <para>
+ If a <emphasis>Structure</emphasis> attribute of
+ <emphasis>Phrase</emphasis> is used in conjunction with a
+ <emphasis>Completeness</emphasis> attribute of
+ <emphasis>Complete (Sub)field</emphasis>, the term is matched
+ against the contents of the phrase (long word) register, if one
+ exists for the given <emphasis>Use</emphasis> attribute.
+ A phrase register is created for those fields in the
+ &acro.grs1; <filename>*.abs</filename> file that contains a
+ <literal>p</literal>-specifier.
<screen>
- Z> scan @attr 1=Title @attr 4=1 @attr 6=3 beethoven
+ Z> scan @attr 1=Title @attr 4=1 @attr 6=3 beethoven
...
bayreuther festspiele (1)
* beethoven bibliography database (1)
benny carter (1)
...
- Z> find @attr 1=Title @attr 4=1 @attr 6=3 "beethoven bibliography"
+ Z> find @attr 1=Title @attr 4=1 @attr 6=3 "beethoven bibliography"
...
Number of hits: 0, setno 5
...
- Z> find @attr 1=Title @attr 4=1 @attr 6=3 "beethoven bibliography database"
+ Z> find @attr 1=Title @attr 4=1 @attr 6=3 "beethoven bibliography database"
...
Number of hits: 1, setno 6
- </screen>
- </para>
+ </screen>
+ </para>
- <para>
- If <emphasis>Structure</emphasis>=<emphasis>Phrase</emphasis> is
- used in conjunction with <emphasis>Incomplete Field</emphasis> - the
- default value for <emphasis>Completeness</emphasis>, the
- search is directed against the normal word registers, but if the term
- contains multiple words, the term will only match if all of the words
- are found immediately adjacent, and in the given order.
- The word search is performed on those fields that are indexed as
- type <literal>w</literal> in the &acro.grs1; <filename>*.abs</filename> file.
+ <para>
+ If <emphasis>Structure</emphasis>=<emphasis>Phrase</emphasis> is
+ used in conjunction with <emphasis>Incomplete Field</emphasis> - the
+ default value for <emphasis>Completeness</emphasis>, the
+ search is directed against the normal word registers, but if the term
+ contains multiple words, the term will only match if all of the words
+ are found immediately adjacent, and in the given order.
+ The word search is performed on those fields that are indexed as
+ type <literal>w</literal> in the &acro.grs1; <filename>*.abs</filename> file.
<screen>
- Z> scan @attr 1=Title @attr 4=1 @attr 6=1 beethoven
+ Z> scan @attr 1=Title @attr 4=1 @attr 6=1 beethoven
...
- beefheart (1)
+ beefheart (1)
* beethoven (18)
- beethovens (7)
+ beethovens (7)
...
- Z> find @attr 1=Title @attr 4=1 @attr 6=1 beethoven
+ Z> find @attr 1=Title @attr 4=1 @attr 6=1 beethoven
...
Number of hits: 18, setno 1
...
...
Number of hits: 2, setno 2
...
- </screen>
- </para>
+ </screen>
+ </para>
- <para>
- If the <emphasis>Structure</emphasis> attribute is
- <emphasis>Word List</emphasis>,
- <emphasis>Free-form Text</emphasis>, or
- <emphasis>Document Text</emphasis>, the term is treated as a
- natural-language, relevance-ranked query.
- This search type uses the word register, i.e. those fields
- that are indexed as type <literal>w</literal> in the
- &acro.grs1; <filename>*.abs</filename> file.
- </para>
+ <para>
+ If the <emphasis>Structure</emphasis> attribute is
+ <emphasis>Word List</emphasis>,
+ <emphasis>Free-form Text</emphasis>, or
+ <emphasis>Document Text</emphasis>, the term is treated as a
+ natural-language, relevance-ranked query.
+ This search type uses the word register, i.e. those fields
+ that are indexed as type <literal>w</literal> in the
+ &acro.grs1; <filename>*.abs</filename> file.
+ </para>
- <para>
- If the <emphasis>Structure</emphasis> attribute is
- <emphasis>Numeric String</emphasis> the term is treated as an integer.
- The search is performed on those fields that are indexed
- as type <literal>n</literal> in the &acro.grs1;
+ <para>
+ If the <emphasis>Structure</emphasis> attribute is
+ <emphasis>Numeric String</emphasis> the term is treated as an integer.
+ The search is performed on those fields that are indexed
+ as type <literal>n</literal> in the &acro.grs1;
<filename>*.abs</filename> file.
- </para>
+ </para>
- <para>
- If the <emphasis>Structure</emphasis> attribute is
- <emphasis>URX</emphasis> the term is treated as a URX (URL) entity.
- The search is performed on those fields that are indexed as type
- <literal>u</literal> in the <filename>*.abs</filename> file.
- </para>
+ <para>
+ If the <emphasis>Structure</emphasis> attribute is
+ <emphasis>URX</emphasis> the term is treated as a URX (URL) entity.
+ The search is performed on those fields that are indexed as type
+ <literal>u</literal> in the <filename>*.abs</filename> file.
+ </para>
- <para>
- If the <emphasis>Structure</emphasis> attribute is
- <emphasis>Local Number</emphasis> the term is treated as
- native &zebra; Record Identifier.
- </para>
+ <para>
+ If the <emphasis>Structure</emphasis> attribute is
+ <emphasis>Local Number</emphasis> the term is treated as
+ native &zebra; Record Identifier.
+ </para>
- <para>
- If the <emphasis>Relation</emphasis> attribute is
- <emphasis>Equals</emphasis> (default), the term is matched
- in a normal fashion (modulo truncation and processing of
- individual words, if required).
- If <emphasis>Relation</emphasis> is <emphasis>Less Than</emphasis>,
- <emphasis>Less Than or Equal</emphasis>,
- <emphasis>Greater than</emphasis>, or <emphasis>Greater than or
- Equal</emphasis>, the term is assumed to be numerical, and a
- standard regular expression is constructed to match the given
- expression.
- If <emphasis>Relation</emphasis> is <emphasis>Relevance</emphasis>,
- the standard natural-language query processor is invoked.
- </para>
+ <para>
+ If the <emphasis>Relation</emphasis> attribute is
+ <emphasis>Equals</emphasis> (default), the term is matched
+ in a normal fashion (modulo truncation and processing of
+ individual words, if required).
+ If <emphasis>Relation</emphasis> is <emphasis>Less Than</emphasis>,
+ <emphasis>Less Than or Equal</emphasis>,
+ <emphasis>Greater than</emphasis>, or <emphasis>Greater than or
+ Equal</emphasis>, the term is assumed to be numerical, and a
+ standard regular expression is constructed to match the given
+ expression.
+ If <emphasis>Relation</emphasis> is <emphasis>Relevance</emphasis>,
+ the standard natural-language query processor is invoked.
+ </para>
- <para>
- For the <emphasis>Truncation</emphasis> attribute,
- <emphasis>No Truncation</emphasis> is the default.
- <emphasis>Left Truncation</emphasis> is not supported.
- <emphasis>Process # in search term</emphasis> is supported, as is
- <emphasis>Regxp-1</emphasis>.
- <emphasis>Regxp-2</emphasis> enables the fault-tolerant (fuzzy)
- search. As a default, a single error (deletion, insertion,
- replacement) is accepted when terms are matched against the register
- contents.
- </para>
+ <para>
+ For the <emphasis>Truncation</emphasis> attribute,
+ <emphasis>No Truncation</emphasis> is the default.
+ <emphasis>Left Truncation</emphasis> is not supported.
+ <emphasis>Process # in search term</emphasis> is supported, as is
+ <emphasis>Regxp-1</emphasis>.
+ <emphasis>Regxp-2</emphasis> enables the fault-tolerant (fuzzy)
+ search. As a default, a single error (deletion, insertion,
+ replacement) is accepted when terms are matched against the register
+ contents.
+ </para>
- </section>
+ </section>
</section>
<section id="querymodel-regular">
<title>&zebra; Regular Expressions in Truncation Attribute (type = 5)</title>
-
+
<para>
Each term in a query is interpreted as a regular expression if
the truncation value is either <emphasis>Regxp-1 (@attr 5=102)</emphasis>
</row>
</tbody>
</tgroup>
- </table>
+ </table>
<para>
The above operands can be combined with the following operators:
</para>
-
+
<table id="querymodel-regular-operators-table" frame="top">
<title>Regular Expression Operators</title>
<tgroup cols="2">
<tbody>
<row>
<entry><literal>x*</literal></entry>
- <entry>Matches <literal>x</literal> zero or more times.
+ <entry>Matches <literal>x</literal> zero or more times.
Priority: high.</entry>
</row>
<row>
<entry><literal>x+</literal></entry>
- <entry>Matches <literal>x</literal> one or more times.
+ <entry>Matches <literal>x</literal> one or more times.
Priority: high.</entry>
</row>
<row>
<entry><literal>x?</literal></entry>
- <entry> Matches <literal>x</literal> zero or once.
+ <entry> Matches <literal>x</literal> zero or once.
Priority: high.</entry>
</row>
<row>
<entry>The order of evaluation may be changed by using parentheses.</entry>
</row>
</tbody>
- </tgroup>
- </table>
-
+ </tgroup>
+ </table>
+
<para>
If the first character of the <literal>Regxp-2</literal> query
is a plus character (<literal>+</literal>) it marks the
beginning of a section with non-standard specifiers.
The next plus character marks the end of the section.
Currently &zebra; only supports one specifier, the error tolerance,
- which consists one digit.
+ which consists one digit.
<!-- TODO Nice thing, but what does
that error tolerance digit *mean*? Maybe an example would be nice? -->
</para>
</para>
</section>
-
+
<!--
<para>
- The RecordType parameter in the <literal>zebra.cfg</literal> file, or
- the <literal>-t</literal> option to the indexer tells &zebra; how to
- process input records.
- Two basic types of processing are available - raw text and structured
- data. Raw text is just that, and it is selected by providing the
- argument <literal>text</literal> to &zebra;. Structured records are
- all handled internally using the basic mechanisms described in the
- subsequent sections.
- &zebra; can read structured records in many different formats.
- </para>
+ The RecordType parameter in the <literal>zebra.cfg</literal> file, or
+ the <literal>-t</literal> option to the indexer tells &zebra; how to
+ process input records.
+ Two basic types of processing are available - raw text and structured
+ data. Raw text is just that, and it is selected by providing the
+ argument <literal>text</literal> to &zebra;. Structured records are
+ all handled internally using the basic mechanisms described in the
+ subsequent sections.
+ &zebra; can read structured records in many different formats.
+ </para>
-->
</section>
<para>
Using the
<literal><cql2rpn>l2rpn.txt</cql2rpn></literal>
- &yaz; Frontend Virtual
+ &yaz; Frontend Virtual
Hosts option, one can configure
the &yaz; Frontend &acro.cql;-to-&acro.pqf;
- converter, specifying the interpretation of various
+ converter, specifying the interpretation of various
<ulink url="&url.cql;">&acro.cql;</ulink>
indexes, relations, etc. in terms of Type-1 query attributes.
- <!-- The yaz-client config file -->
+ <!-- The yaz-client config file -->
</para>
<para>
For example, using server-side &acro.cql;-to-&acro.pqf; conversion, one might
query a zebra server like this:
<screen>
- <![CDATA[
+ <![CDATA[
yaz-client localhost:9999
Z> querytype cql
Z> find text=(plant and soil)
]]>
</screen>
- and - if properly configured - even static relevance ranking can
- be performed using &acro.cql; query syntax:
+ and - if properly configured - even static relevance ranking can
+ be performed using &acro.cql; query syntax:
<screen>
- <![CDATA[
+ <![CDATA[
Z> find text = /relevant (plant and soil)
]]>
- </screen>
+ </screen>
</para>
<para>
- By the way, the same configuration can be used to
+ By the way, the same configuration can be used to
search using client-side &acro.cql;-to-&acro.pqf; conversion:
- (the only difference is <literal>querytype cql2rpn</literal>
- instead of
+ (the only difference is <literal>querytype cql2rpn</literal>
+ instead of
<literal>querytype cql</literal>, and the call specifying a local
conversion file)
<screen>
- <![CDATA[
+ <![CDATA[
yaz-client -q local/cql2pqf.txt localhost:9999
Z> querytype cql2rpn
Z> find text=(plant and soil)
]]>
- </screen>
+ </screen>
</para>
<para>
Exhaustive information can be found in the
Section <ulink url="&url.yaz.cql2pqf;">&acro.cql; to &acro.rpn; conversion</ulink>
in the &yaz; manual.
- </para>
- <!--
- <para>
- See
+ </para>
+ <!--
+ <para>
+ See
<ulink url="http://www.loc.gov/z3950/agency/zing/cql/dc-indexes.html"/>
for the Maintenance Agency's work-in-progress mapping of Dublin Core
- indexes to Attribute Architecture (util, XD and BIB-2)
+ indexes to Attribute Architecture (util, XD and BIB-2)
attributes.
</para>
-->
- </section>
+ </section>
-</chapter>
+ </chapter>
<!-- Keep this comment at the end of the file
Local variables:
sgml-always-quote-attributes:t
sgml-indent-step:1
sgml-indent-data:t
- sgml-parent-document: "zebra.xml"
+ sgml-parent-document: "idzebra.xml"
sgml-local-catalogs: nil
sgml-namecase-general:t
End:
projects / idzebra-moved-to-github.git / blobdiff