<chapter id="introduction">
- <!-- $Id: introduction.xml,v 1.11 2002-08-29 14:05:11 mike Exp $ -->
+ <!-- $Id: introduction.xml,v 1.26 2003-10-30 11:11:57 adam Exp $ -->
<title>Introduction</title>
<sect1>
<title>Overview</title>
<para>
- <ulink url="http://www.indexdata.dk/zebra/">
- Zebra</ulink>
+ <ulink url="http://indexdata.dk/zebra/">Zebra</ulink>
is a high-performance, general-purpose structured text
- indexing and retrieval engine. It reads structured records in a
+ indexing and retrieval engine. It reads records in a
variety of input formats (eg. email, XML, MARC) and provides access
to them through a powerful combination of boolean search
expressions and relevance-ranked free-text queries.
and how to configure the server to give you the
functionality that you need.
</para>
-
- <para>
- If you use Zebra, you should visit its
- <ulink url="http://www.indexdata.dk/zebra/">web site</ulink>,
- where you can join the
- <ulink url="http://www.indexdata.dk/mailman/listinfo/zebralist">
- mailing-list</ulink>
- by sending email to
- <email>### zebra-subscribe@mailman.indexdata.dk</email>
- </para>
-
</sect1>
<sect1 id="features">
<listitem>
<para>
- Very large databases: files for indexes, etc. can be
+ Very large databases: logical files can be
automatically partitioned over multiple disks.
</para>
</listitem>
<listitem>
<para>
Arbitrarily complex records. The internal data format
- is an structured format conceptually similar to XML or GRS-1,
- which allows nested structured data elements and
+ is a structured format conceptually similar to XML or GRS-1,
+ which allows lists, nested structured data elements and
variant forms of data.
</para>
</listitem>
<para>
Configurable to understand many input formats.
A system of input filters driven by
- regular expressions allows you to easily process most ASCII-based
- data formats. SGML, XML, ISO2709 (MARC), and raw text are also
+ regular expressions allows most ASCII-based
+ data formats to be easily processed.
+ SGML, XML, ISO2709 (MARC), and raw text are also
supported.
</para>
</listitem>
Searching supports a powerful combination of boolean queries as
well as relevance-ranking (free-text) queries. Truncation,
masking, full regular expression matching and "approximate
- matching" (eg. spelling mistakes) are all supported.
+ matching" (eg. spelling mistakes) are all handled.
</para>
</listitem>
<para>
Zebra is written in portable C, so it runs on most Unix-like systems
as well as Windows NT. A binary distribution for Windows NT is
- available.
+ available at
+ <ulink url="http://ftp.indexdata.dk/pub/zebra/win32/"/>,
+ and pre-built packages are available for some Linux
+ distributions:
+ Red Hat 7.x RPMs at
+ <ulink url="http://ftp.indexdata.dk/pub/zebra/RedHat7.X/"/>
+ and Debian packages at
+ <ulink url="http://ftp.indexdata.dk/pub/zebra/debian/"/>
</para>
</listitem>
<itemizedlist>
<listitem>
<para>
- Protocol facilities: Init, Search, Present (retrieval), Delete,
- Scan (index browsing) and Sort.
+ Protocol facilities: Init, Search, Present (retrieval),
+ Segmentation (support for very large records), Delete, Scan
+ (index browsing), Sort, Close and support for the ``update''
+ Extended Service to add or replace an existing XML record.
+ <!-- Adam says:
+ * Supported
+ You can insert/delete/replace an XML record given an
+ "external" ID. Actually this way of doing ES Update was
+ meant for an OAI application that Ian Ibbotson had in
+ mind to implement. The "update" command in YAZ client
+ implements this on the client side. My plan is to make
+ this available in ZOOM "extended" soon..
+ -->
</para>
</listitem>
<listitem>
<para>
- Piggy-backed presents are honored in the search-request.
+ Piggy-backed presents are honored in the search request - that
+ is, a subset of the found records can be returned directly with
+ a search response, enabling search and retrieval to happen in a
+ single round-trip.
</para>
</listitem>
</sect1>
- <sect1 id="apps">
+ <sect1 id="apps">
<title>Applications</title>
<para>
Zebra has been deployed in numerous applications, in both the
</sect2>
<sect2>
+ <title>NLI-Z39.50 - a Natural Language Interface for Libraries</title>
+ <para>
+ Fernuniversität Hagen in Germany have developed a natural
+ language interface for access to library databases.
+ <ulink url="http://ki212.fernuni-hagen.de/nli/NLIintro.html"/>
+ In order to evaluate this interface for recall and precision, they
+ chose Zebra as the basis for retrieval effectiveness. The Zebra
+ server contains a copy of the GIRT database, consisting of more
+ than 76000 records in SGML format (bibliographic records from
+ social science), which are mapped to MARC for presentation.
+ </para>
+ <para>
+ (GIRT is the German Indexing and Retrieval Testdatabase. It is a
+ standard German-language test database for intelligent indexing
+ and retrieval systems. See
+ <ulink url="http://www.gesis.org/forschung/informationstechnologie/clef-delos.htm"/>)
+ </para>
+ <para>
+ Evaluation will take place as part of the TREC/CLEF campaign 2003
+ <ulink url="http://clef.iei.pi.cnr.it or http://www4.eurospider.ch/CLEF/"/>
+ </para>
+ <para>
+ For more information, contact Johannes Leveling
+ <email>Johannes.Leveling@FernUni-Hagen.De</email>
+ </para>
+ </sect2>
+
+ <sect2>
+ <title>ULS (Union List of Serials)</title>
+ <para>
+ The M25 Systems Team
+ has created a union catalogue for the periodicals of the
+ twenty-one constituent libraries of the University of London and
+ the University of Westminster
+ (<ulink url="http://www.m25lib.ac.uk/ULS/"/>).
+ They have achieved this using an
+ unusual architecture, which they describe as a
+ ``non-distributed virtual union catalogue''.
+ </para>
+ <para>
+ The member libraries send in data files representing their
+ periodicals, including both brief bibliographic data and summary
+ holdings. Then 21 individual Z39.50 targets are created, each
+ using Zebra, and all mounted on the single hardware server.
+ The live service provides a web gateway allowing Z39.50 searching
+ of all of the targets or a selection of them. Zebra's small
+ footprint allows a relatively modest system to comfortably host
+ the 21 servers.
+ </para>
+ <para>
+ More information can be found at
+ <ulink url="http://www.m25lib.ac.uk/ULS/"/>
+ </para>
+ </sect2>
+
+ <sect2>
<title>Various web indexes</title>
<para>
Zebra has been used by a variety of institutions to construct
indexes of large web sites, typically in the region of tens of
millions of pages. In this role, it functions somewhat similarly
to the engine of google or altavista, but for a selected intranet
- or subset of the whole Web.
+ or a subset of the whole Web.
+ </para>
+ <para>
+ For example, Liverpool University's web-search facility (see on
+ the home page at
+ <ulink url="http://www.liv.ac.uk/"/>
+ and many sub-pages) works by relevance-searching a Zebra database
+ which is populated by the Harvest-NG web-crawling software.
</para>
<para>
- ### examples, details and numbers, please!
+ For more information on Liverpool university's intranet search
+ architecture, contact John Gilbertson
+ <email>jgilbert@liverpool.ac.uk</email>
+ </para>
+ <para>
+ Kang-Jin Lee
+ <email>lee@arco.de</email>,
+ has recently modified the Harvest web indexer to use Zebra as
+ its native repository engine. His comments on the switch over
+ from the old engine are revealing:
+ <blockquote>
+ <para>
+ The first results after some testing with Zebra are very
+ promising. The tests were done with around 220,000 SOIF files,
+ which occupies 1.6GB of disk space.
+ </para>
+ <para>
+ Building the index from scratch takes around one hour with Zebra
+ where [old-engine] needs around five hours. While [old-engine]
+ blocks search requests when updating its index, Zebra can still
+ answer search requests.
+ [...]
+ Zebra supports incremental indexing which will speed up indexing
+ even further.
+ </para>
+ <para>
+ While the search time of [old-engine] varies from some seconds
+ to some minutes depending how expensive the query is, Zebra
+ usually takes around one to three seconds, even for expensive
+ queries.
+ [...]
+ Zebra can search more than 100 times faster than [old-engine]
+ and can process multiple search requests simultaneously
+ </para>
+ <para>
+ I am very happy to see such nice software available under GPL.
+ </para>
+ </blockquote>
</para>
</sect2>
</sect1>
+
+ <sect1 id="support">
+ <title>Support</title>
+ <para>
+ You can get support for Zebra from at least three sources.
+ </para>
+ <para>
+ First, there's the Zebra web site at
+ <ulink url="http://indexdata.dk/zebra/"/>,
+ which always has the most recent version available for download.
+ If you have a problem with Zebra, the first thing to do is see
+ whether it's fixed in the current release.
+ </para>
+ <para>
+ Second, there's the Zebra mailing list. Its home page at
+ <ulink url="http://indexdata.dk/mailman/listinfo/zebralist"/>
+ includes a complete archive of all messages that have ever been
+ posted on the list. The Zebra mailing list is used both for
+ announcements from the authors (new
+ releases, bug fixes, etc.) and general discussion. You are welcome
+ to seek support there. Join by sending email to
+ <email>zebra-request@indexdata.dk</email> with the word
+ <literal>subscribe</literal> in the body of the message.
+ </para>
+ <para>
+ Third, it's possible to buy a commercial support contract, with
+ well defined service levels and response times, from Index Data.
+ See
+ <ulink url="http://indexdata.dk/support/"/>
+ for details.
+ </para>
+ </sect1>
+
+
<sect1 id="future">
<title>Future Directions</title>
Improved support for XML in search and retrieval. Eventually,
the goal is for Zebra to pull double duty as a flexible
information retrieval engine and high-performance XML
- repository.
+ repository. The recent addition of XPath searching is one
+ example of the kind of enhancement we're working on.
</para>
</listitem>
<listitem>
<para>
- Access to search engine through SOAP/RPC API to allow the
+ Access to the search engine through SOAP/RPC API to allow the
construction of applications without requiring Z39.50 tools.
+ This will shortly be available by means of Index Data's
+ SRW-to-Z39.50 gateway, currently in beta test.
</para>
</listitem>
<listitem>
<para>
+ Support for the use of Perl both for access to the Zebra API
+ and for building extension ``plug-ins'' such as input filters.
+ The code for this has been contributed to the source tree by
+ Peter Popovics
+ <email>pop@technomat.hu</email>,
+ and is in the process of being integrated and tested.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
Improved free-text searching. We're first and foremost octet jockeys and
we're actively looking for organisations or people who'd like
to contribute experience in relevance ranking and text
</para>
<para>
If you think it's all really neat, you're welcome to drop us a line
- saying that, too. You'll find contact info at the end of this file.
+ saying that, too. You can email us on
+ <email>info@indexdata.dk</email>
+ or check the contact info at the end of this manual.
</para>
</sect1>
projects / idzebra-moved-to-github.git / blobdiff