<chapter id="examples">
- <!-- $Id: examples.xml,v 1.3 2002-08-30 01:17:10 mike Exp $ -->
+ <!-- $Id: examples.xml,v 1.11 2002-10-17 21:54:22 mike Exp $ -->
<title>Example Configurations</title>
<sect1>
<listitem>
<para>
- Where to find the default indexing rules (### default.idx)
+ Where to find subsidiary configuration files, including
+ <literal>default.idx</literal>
+ which specifies the default indexing rules.
</para>
</listitem>
<listitem>
<para>
- ### Something to do with explain.abs?!
+ What attribute sets to recognise in searches.
</para>
</listitem>
<listitem>
<para>
- ### Where to find other configuration files, e.g. searches using
- BIB-1 attributes require a bib1.att configuration file (even if
- the access point is actually an XPath expression). These are
- searched for in the working directory unless otherwise
- specified.
+ Policy details such as what record type to expect, what
+ low-level indexing algorithm to use, how to identify potential
+ duplicate records, etc.
</para>
</listitem>
</itemizedlist>
</para>
+ <para>
+ Now let's see what goes in the <literal>zebra.cfg</literal> file
+ for some example configurations.
+ </para>
</sect1>
- <sect1>
- <title>Example 1: Minimal Configuration</title>
+ <sect1 id="example1">
+ <title>Example 1: XML Indexing And Searching</title>
<para>
This example shows how Zebra can be used with absolutely minimal
- configuration to index a body of XML documents, and search them
- using XPath expressions to specify access points.
+ configuration to index a body of
+ <ulink url="http://www.w3.org/xml/###">XML</ulink>
+ documents, and search them using
+ <ulink url="http://www.w3.org/xpath/###">XPath</ulink>
+ expressions to specify access points.
</para>
<para>
- Go to the <literal>zebra/examples/dinosauricon</literal> directory.
+ Go to the <literal>examples/dinosauricon</literal> subdirectory
+ of the distribution archive.
There you will find a <literal>records</literal> subdirectory,
which contains some raw XML data to be added to the database: in
- this case, two files, <literal>genera.xml</literal> and
- <literal>taxa.xml</literal>, which contain information about all
- the known dinosaur genera as of August 2002.
+ this case, as single file, <literal>genera.xml</literal>,
+ which contain information about all the known dinosaur genera as of
+ August 2002.
</para>
<para>
Now we need to create the Zebra database, which we do with the
- Zebra indexer, <literal>zebraidx</literal>. This program's
- behaviour is driven by a configuration life, generally called
- <literal>zebra.cfg</literal>, although this can be changed with the
- <literal>-c</literal> option. For our purposes, we don't need any
- special behaviour - we can use the defaults - so an empty
- configuration will do just fine. We can either create an empty
- <literal>zebra.cfg</literal> or specify the name of an existing
- empty file using, for example, <literal>-c /dev/null</literal>.
- </para>
- <para>
- In this case, we'll use an empty <literal>zebra.cfg</literal> so
- we can add more configuration to it later.
+ Zebra indexer, <literal>zebraidx</literal>, which is
+ driven by the <literal>zebra.cfg</literal> configuration file.
+ For our purposes, we don't need any
+ special behaviour - we can use the defaults - so we start with a
+ minimal file that just tells <literal>zebraidx</literal> where to
+ find the default indexing rules, and how to parse the records:
+ <screen>
+ profilePath: .:../../tab:../../../yaz/tab
+ recordType: grs.sgml
+ </screen>
</para>
<para>
That's all you need for a minimal Zebra configuration. Now you can
roll the XML records into the database and build the indexes:
<screen>
- zebraidx -t grs.sgml update records
+ zebraidx update records
</screen>
- (### What does "grs.sgml" actually mean?)
</para>
<para>
Now start the server. Like the indexer, its behaviour is
- controlled by a configuration file, generally
- <literal>zebra.cfg</literal>; and like the indexer, it works just
- fine with an empty configuration.
+ controlled by the
+ <literal>zebra.cfg</literal> file; and like the indexer, it works
+ just fine with this minimal configuration.
<screen>
zebrasrv
</screen>
By default, the server listens on IP port number 9999, although
- this can easily be changed.
+ this can easily be changed - see
+ <xref linkend="zebrasrv"/>.
</para>
<para>
Now you can use the Z39.50 client program of your choice to execute
XPath-based boolean queries and fetch the XML records that satisfy
them:
<screen>
- Z> open tcp:@:9999
- Connecting...Ok.
- Z> find @attr 1=/GENUS/MEANING @or vertebra jaw
- Number of hits: 1
- Z> format xml
- Z> show 1
- Z> show 1
- <GENUS name="Hudiesaurus" type="with" xmlns:idzebra="http://www.indexdata.dk/zebra/">
- <MEANING>
- butterfly <LOW>vertebra</LOW> lizard
- </MEANING>
- <LENGTH value="30"></LENGTH>
- <PLACE name="China"></PLACE>
- <REMAINS content="4 teeth, forelimb, first dorsal vertebra"></REMAINS>
- <SPECIES name="sinojapanorum" status="nudum">
- <AUTHOR name="Dong" year="1997"></AUTHOR>
- <MEANING>
- Chinese-Japanese
- </MEANING>
- </SPECIES>
- <idzebra:size>359</idzebra:size><idzebra:localnumber>447</idzebra:localnumber><idzebra:filename>records/genera.xml</idzebra:filename></GENUS>
+ $ yaz-client tcp:@:9999
+ Connecting...Ok.
+ Z> find @attr 1=/GENUS/SPECIES/AUTHOR/@name Wedel
+ Number of hits: 1
+ Z> format xml
+ Z> show 1
+ <GENUS name="Sauroposeidon" type="with">
+ <MEANING>lizard Poseidon <LOW>(Greek god of, among other things, earthquakes)</LOW></MEANING>
+ <SPECIES name="proteles">
+ <AUTHOR type="vide" name="Franklin" year="2000"></AUTHOR>
+ <AUTHOR name="Wedel, Cifelli, Sanders"></AUTHOR>
+ </SPECIES>
+ <PLACE name="Oklahoma"></PLACE>
+ <TIME value="Albian"></TIME>
+ <LENGTH value="30" q="1"></LENGTH>
+ <REMAINS content="rib, cervical vertebrae"></REMAINS>
+ <ESSAY>
+ <P> This new <NOMEN name="Brachiosaurus"></NOMEN>-like <LINK content="dinosaur"></LINK>
+ was perhaps the tallest. With its head raised, it stood 60 feet (nearly
+ 20 m) tall. </P>
+ </ESSAY>
+
+ <idzebra xmlns="http://www.indexdata.dk/zebra/">
+ <size>593</size>
+ <localnumber>891</localnumber>
+ <filename>records/genera.xml</filename>
+ </idzebra>
+ </GENUS>
</screen>
</para>
<para>
</para>
</sect1>
- <sect1>
- <title>Example 2: Adding Some Configuration</title>
+ <sect1 id="example2">
+ <title>Example 2: Supporting Interoperable Searches</title>
+
+ <para>
+ The problem with the previous example is that you need to know the
+ structure of the documents in order to find them. For example,
+ when we wanted to know the genera for which Matt Wedel is an
+ author
+ (<phrase role="taxon">Sauroposeidon proteles</phrase>),
+ we had to formulate a complex XPath
+ <literal>1=/GENUS/SPECIES/AUTHOR/@name</literal>
+ which embodies the knowledge that author names are specified in the
+ <literal>name</literal> attribute of the
+ <literal><AUTHOR></literal> element,
+ which is inside the
+ <literal><SPECIES></literal> element,
+ which in turn is inside the top-level
+ <literal><GENUS></literal> element.
+ </para>
+ <para>
+ This is bad not just because it requires a lot of typing, but more
+ significantly because it ties searching semantics to the physical
+ structure of the searched records. You can't use the same search
+ specification to search two databases if their internal
+ representations are different. Consider an alternative dinosaur
+ database in which the records have author names specified
+ inside an <literal><authorName></literal> element directly
+ inside a top-level <literal><taxon></literal> element: then
+ you'd need to search for them using
+ <literal>1=/taxon/authorName</literal>
+ </para>
+ <para>
+ How, then, can we build broadcasting Information Retrieval
+ applications that look for records in many different databases?
+ The Z39.50 protocol offers a powerful and general solution to this:
+ abstract ``access points''. In the Z39.50 model, an access point
+ is simply a point at which searches can be directed. Nothing is
+ said about implementation: in a given database, an access point
+ might be implemented as an index, a path into physical records, an
+ algorithm for interrogating relational tables or whatever works.
+ The key point is that the semantics of an access point are fixed
+ and well defined.
+ </para>
+ <para>
+ For convenience, access points are gathered into <firstterm>attribute
+ sets</firstterm>. For example, the BIB-1 attribute set is supposed to
+ contain bibliographic access points such as author, title, subject
+ and ISBN; the GEO attribute set contains access points pertaining
+ to geospatial information (bounding box, ###, etc.); the CIMI
+ attribute set contains access points to do with museum collections
+ (provenance, inscriptions, etc.)
+ </para>
+ <para>
+ In practice, the BIB-1 attribute set has tended to be a dumping
+ ground for all sorts of access points, so that, for example, it
+ includes some geospatial access points as well as strictly
+ bibliographic ones. Nevertheless, the key point is that this model
+ allows a layer of abstraction over the physical representation of
+ records in databases.
+ </para>
+ <para>
+ In the BIB-1 attribute set, an author search is represented by
+ access point 1003. (See
+ <ulink url="###bib1-semantics"/>)
+ So we need to configure our dinosaur database so that searches for
+ BIB-1 access point 1003 look the
+ <literal>name</literal> attribute of the
+ <literal><AUTHOR></literal> element,
+ inside the
+ <literal><SPECIES></literal> element,
+ inside the top-level
+ <literal><GENUS></literal> element.
+ </para>
+ <para>
+ This is a two-step process. First, we need to tell Zebra that we
+ want to support the BIB-1 attribute set. Then we need to tell it
+ which elements of its record pertain to access point 1003.
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ </para>
+ </listitem>
+ </itemizedlist>
+ </sect1>
+</chapter>
+
+
+<!--
<para>
You may have noticed as <literal>zebraidx</literal> was building
- the database that it issued several warnings, which we ignored at
- the time:
- <screen>
-zebraidx -t grs.sgml update records
-02:12:32-30/08: zebraidx(18151) [warn] default.idx [No such file or directory]
-02:12:32-30/08: zebraidx(18151) [warn] Couldn't open explain.abs [No such file or directory]
-02:12:32-30/08: zebraidx(18151) [warn] records/genera.xml:0 Couldn't open GENUS.abs [No such file or directory]
-02:12:32-30/08: zebraidx(18151) [warn] records/genera.xml:0 Unknown register type: 0
-02:12:32-30/08: zebraidx(18151) [warn] records/genera.xml:0 Unknown register type: w
-02:12:35-30/08: zebraidx(18151) [warn] records/taxa.xml:0 Couldn't open TAXON.abs [No such file or directory]
- </screen>
- And the server issued several more as the client connected to it,
- then searched for and retrieved a record:
+ the database that it issued a warning, which we ignored at the
+ time:
<screen>
-02:17:10-30/08: zebrasrv(18165) [warn] default.idx [No such file or directory]
-02:17:10-30/08: zebrasrv(18165) [warn] Couldn't open explain.abs [No such file or directory]
-02:17:57-30/08: zebrasrv(18165) [warn] Unknown register type: w
-02:18:42-30/08: zebrasrv(18165) [warn] Couldn't open GENUS.abs [No such file or directory]
+ $ zebraidx update records
+ 00:45:46-08/10: ../../index/zebraidx(5016) [warn] records/genera.xml:0 Couldn't open GENUS.abs [No such file or directory]
</screen>
+ FIXME ### This needs more text
</para>
- </sect1>
-</chapter>
+-->
<!--
The master configuration file, <literal>zebra.cfg</literal>,
which is as short and simple as it can be:
<screen>
- # $Header: /home/cvsroot/idis/doc/examples.xml,v 1.3 2002-08-30 01:17:10 mike Exp $
+ # $Header: /home/cvsroot/idis/doc/examples.xml,v 1.11 2002-10-17 21:54:22 mike Exp $
# Bare-bones master configuration file for Zebra
profilePath: .:../../tab:../../../yaz/tab
</screen>
### What is an attribute set?
</para>
</listitem>
+-->
+
+<!--
+ The simplest hello-world example could go like this:
+
+ Index the document
+
+ <book>
+ <title>The art of motorcycle maintenance</title>
+ <subject scheme="Dewey">zen</subject>
+ </book>
+
+ And search it like
+
+ f @attr 1=/book/title motorcycle
+
+ f @attr 1=/book/subject[@scheme=Dewey] zen
+
+ If you suddenly decide you want broader interop, you can add
+ an abs file (more or less like this):
+
+ attset bib1.att
+ tagset tagsetg.tag
+
+ elm (2,1) title title
+ elm (2,21) subject subject
+-->
+
+<!--
+How to include images:
+
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="system.eps" format="eps">
+ </imageobject>
+ <imageobject>
+ <imagedata fileref="system.gif" format="gif">
+ </imageobject>
+ <textobject>
+ <phrase>The Multi-Lingual Search System Architecture</phrase>
+ </textobject>
+ <caption>
+ <para>
+ <emphasis role="strong">
+ The Multi-Lingual Search System Architecture.
+ </emphasis>
+ <para>
+ Network connections across local area networks are
+ represented by straight lines, and those over the
+ internet by jagged lines.
+ </caption>
+ </mediaobject>
+
+Whene the three <*object> thingies inside the top-level <mediaobject>
+are decreasingly preferred version to include depending on what the
+rendering engine can handle. I generated the EPS version of the image
+by exporting a line-drawing done in TGIF, then converted that to the
+GIF using a shell-script called "epstogif" which used an appallingly
+baroque sequence of conversions, which I would prefer not to pollute
+the Zebra build environment with:
+
+ #!/bin/sh
+
+ # Yes, what follows is stupidly convoluted, but I can't find a
+ # more straightforward path from the EPS generated by tgif's
+ # "Print" command into a browser-friendly format.
+
+ file=`echo "$1" | sed 's/\.eps//'`
+ ps2pdf "$1" "$file".pdf
+ pdftopbm "$file".pdf "$file"
+ pnmscale 0.50 < "$file"-000001.pbm | pnmcrop | ppmtogif
+ rm -f "$file".pdf "$file"-000001.pbm
- <listitem>
- <para>
- The BIB-1 attribute set configuration file,
- <literal>bib1.att</literal>, which is also as short as possible:
- <screen>
- # $Header: /home/cvsroot/idis/doc/examples.xml,v 1.3 2002-08-30 01:17:10 mike Exp $
- # Bare-bones BIB-1 attribute set file for Zebra
- reference Bib-1
- </screen>
- Apart from the comments, all this specifies is that reference of
- the attribute set described by this file is
- <literal>Bib-1</literal>, a name recognised by the system as
- referring to a well-known opaque identifier that is transmitted
- by clients as part of their searches.
- ### Yeuch! Surely we can say that better!
- </para>
- <para>
- ### Can't we somehow say this trivial thing in the main
- configuration file?
- </para>
- </listitem>
-->
<!-- Keep this comment at the end of the file
projects / idzebra-moved-to-github.git / blobdiff