<chapter id="introduction">
- <!-- $Id: introduction.xml,v 1.19 2002-10-20 14:02:03 mike Exp $ -->
+ <!-- $Id: introduction.xml,v 1.33 2006-06-13 13:45:08 marc Exp $ -->
<title>Introduction</title>
<sect1>
<title>Overview</title>
<para>
- <ulink url="http://indexdata.dk/zebra/">
- Zebra</ulink>
+ <ulink url="http://indexdata.dk/zebra/">Zebra</ulink>
is a high-performance, general-purpose structured text
- indexing and retrieval engine. It reads structured records in a
+ indexing and retrieval engine. It reads records in a
variety of input formats (eg. email, XML, MARC) and provides access
to them through a powerful combination of boolean search
expressions and relevance-ranked free-text queries.
<listitem>
<para>
- Very large databases: files for indexes, etc. can be
+ Very large databases: logical files can be
automatically partitioned over multiple disks.
</para>
</listitem>
<listitem>
<para>
Arbitrarily complex records. The internal data format
- is an structured format conceptually similar to XML or GRS-1,
+ is a structured format conceptually similar to XML or GRS-1,
which allows lists, nested structured data elements and
variant forms of data.
</para>
as well as Windows NT. A binary distribution for Windows NT is
available at
<ulink url="http://ftp.indexdata.dk/pub/zebra/win32/"/>,
- and pre-built packages are available for some Linux
+ and pre-built packages are available for
+ <!--- some Linux
distributions:
Red Hat 7.x RPMs at
<ulink url="http://ftp.indexdata.dk/pub/zebra/RedHat7.X/"/>
and Debian packages at
- <ulink url="http://ftp.indexdata.dk/pub/zebra/debian/"/>
+ -->
+ <literal>GNU/Debian Linux</literal> at
+ <ulink url="http://ftp.indexdata.dk/pub/zebra/debian/"/>.
</para>
</listitem>
</para>
<para>
More information can be found at
- <ulink url="http://www.dtv.dk/help/dads/index_e.htm"/>
+ <ulink url="http://www.dtv.dk/"/> and
+ <ulink url="http://dads.dtv.dk"/>
+ </para>
+ </sect2>
+
+ <sect2>
+ <title>Infonet Eprints</title>
+ <para>
+ The InfoNet Eprints service from the
+ <ulink url="http://www.dtv.dk/">
+ Technical Knowledge Center of Denmark</ulink>
+ provides access to documents stored in
+ eprint/preprint servers and institutional research archives around
+ the world. The service is based on Open Archives Initiative metadata
+ harvesting of selected scientific archives around the world. These
+ open archives offer free and unrestricted access to their contents.
+ </para>
+ <para>
+ Infonet Eprints currently holds 1.4 million records from 16 archives.
+ The online search facility is found at
+ <ulink url="http://preprints.cvt.dk"/>.
</para>
</sect2>
<sect2>
<title>NLI-Z39.50 - a Natural Language Interface for Libraries</title>
<para>
- Fernuniversität Hagen in Germany have developed a natural
+ Fernuniversität Hagen in Germany have developed a natural
language interface for access to library databases.
- <ulink url="http://ki212.fernuni-hagen.de/nli/NLIintro.html"/>
+ <!-- <ulink
+ url="http://ki212.fernuni-hagen.de/nli/NLIintro.html"/> -->
In order to evaluate this interface for recall and precision, they
chose Zebra as the basis for retrieval effectiveness. The Zebra
server contains a copy of the GIRT database, consisting of more
</para>
<para>
Evaluation will take place as part of the TREC/CLEF campaign 2003
- <ulink url="http://clef.iei.pi.cnr.it or http://www4.eurospider.ch/CLEF/"/>
+ <ulink url="http://clef.iei.pi.cnr.it"/>.
+ <!-- or <ulink url="http://www4.eurospider.ch/CLEF/"/> -->
</para>
<para>
For more information, contact Johannes Leveling
<sect2>
<title>ULS (Union List of Serials)</title>
<para>
- The M25-Link systems team
- (<ulink url="http://www.m25lib.ac.uk/M25link/"/>)
- are involved in a project called ULS to provide a union catalogue
- for periodicals in 21 member libraries. They do this with an
- unusual architecture which they call a
+ The M25 Systems Team
+ has created a union catalogue for the periodicals of the
+ twenty-one constituent libraries of the University of London and
+ the University of Westminster
+ (<ulink url="http://www.m25lib.ac.uk/ULS/"/>).
+ They have achieved this using an
+ unusual architecture, which they describe as a
``non-distributed virtual union catalogue''.
</para>
<para>
which is populated by the Harvest-NG web-crawling software.
</para>
<para>
- For more information, contact John Gilbertson
+ For more information on Liverpool university's intranet search
+ architecture, contact John Gilbertson
<email>jgilbert@liverpool.ac.uk</email>
</para>
+ <para>
+ Kang-Jin Lee
+ <email>lee@arco.de</email>,
+ has recently modified the Harvest web indexer to use Zebra as
+ its native repository engine. His comments on the switch over
+ from the old engine are revealing:
+ <blockquote>
+ <para>
+ The first results after some testing with Zebra are very
+ promising. The tests were done with around 220,000 SOIF files,
+ which occupies 1.6GB of disk space.
+ </para>
+ <para>
+ Building the index from scratch takes around one hour with Zebra
+ where [old-engine] needs around five hours. While [old-engine]
+ blocks search requests when updating its index, Zebra can still
+ answer search requests.
+ [...]
+ Zebra supports incremental indexing which will speed up indexing
+ even further.
+ </para>
+ <para>
+ While the search time of [old-engine] varies from some seconds
+ to some minutes depending how expensive the query is, Zebra
+ usually takes around one to three seconds, even for expensive
+ queries.
+ [...]
+ Zebra can search more than 100 times faster than [old-engine]
+ and can process multiple search requests simultaneously
+ </para>
+ <para>
+ I am very happy to see such nice software available under GPL.
+ </para>
+ </blockquote>
+ </para>
</sect2>
</sect1>
</para>
<para>
Second, there's the Zebra mailing list. Its home page at
- <ulink url="http://indexdata.dk/mailman/listinfo/zebralist"/>
+ <ulink url="http://lists.indexdata.dk/cgi-bin/mailman/listinfo/zebralist"/>
includes a complete archive of all messages that have ever been
posted on the list. The Zebra mailing list is used both for
announcements from the authors (new
releases, bug fixes, etc.) and general discussion. You are welcome
- to seek support there. Join by sending email to
- <email>zebra-request@indexdata.dk</email>. Put the word
- <literal>subscribe</literal> in the body of the message.
+ to seek support there. Join by filling the form on the list home page.
</para>
<para>
Third, it's possible to buy a commercial support contract, with
well defined service levels and response times, from Index Data.
See
- <ulink url="http://indexdata.dk/support/?lang=en"/>
- <!-- ### compare this page with http://indexdata.dk/support2/ -->
+ <ulink url="http://indexdata.dk/support/"/>
for details.
</para>
</sect1>
Improved support for XML in search and retrieval. Eventually,
the goal is for Zebra to pull double duty as a flexible
information retrieval engine and high-performance XML
- repository.
+ repository. The recent addition of XPath searching is one
+ example of the kind of enhancement we're working on.
</para>
<para>
- ### Partially done.
+ There is also the experimental <literal>ALVIS XSLT</literal>
+ XML input filter, which unleashes the full power of DOM based
+ XSLT transformations during indexing and record retrieval. Work
+ on this filter has been sponsored by the ALVIS EU project
+ <ulink url="http://www.alvis.info/alvis/"/>. We expect this filter to
+ mature soon, as it is planned to be included in the version 1.4
+ release of Zebra.
</para>
</listitem>
<listitem>
<para>
- Access to search engine through SOAP/RPC API to allow the
+ Access to the search engine through SOAP/RPC API to allow the
construction of applications without requiring Z39.50 tools.
- </para>
- <para>
- ### Partially done, thanks to the new SRW/Z39.50 gateway.
+ <!--
+ This will shortly be available by means of Index Data's
+ <ulink url="http://www.loc.gov/standards/sru/srw/">SRW</ulink>-to-Z39.50 gateway, currently in beta test.
+ -->
+ Experimental support of the
+ Search/Retrieve Via URL ( <ulink url="&url.sru;">SRU</ulink>)
+ <ulink url="&url.sru;"/>
+ REST webservice, and the
+ Search/Retrieve Web Service ( <ulink url="http://www.loc.gov/standards/sru/srw/">SRW</ulink>)
+ <ulink url="http://www.loc.gov/standards/sru/srw/"/>
+ SOAP Web Service have recently been added to the YAZ/Zebra
+ combo - including server side Common Query Language (<ulink url="&url.cql;">CQL</ulink>)
+ <ulink url="&url.cql;"/> parsing
+ and configuration. It remains to find a sponsor for further testing,
+ documentation and packaging of this exiting component.
</para>
</listitem>
<listitem>
<para>
+ Support for the use of Perl both for access to the Zebra API
+ and for building extension ``plug-ins'' such as input filters.
+ The code for this has been contributed to the source tree by
+ Peter Popovics
+ <email>pop@technomat.hu</email>,
+ and is in the process of being integrated and tested.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
Improved free-text searching. We're first and foremost octet jockeys and
we're actively looking for organisations or people who'd like
to contribute experience in relevance ranking and text
projects / idzebra-moved-to-github.git / blobdiff