X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=doc%2Fintroduction.xml;h=47096a3c826ef4ceb7d693db1d02080bbc628e0f;hb=b19b79e382ef8196f1625763db1af3a82b1e0c81;hp=29d759dc84706a10dfaa74e2ee2e8fbbb2e909ae;hpb=1b368282fac5d740ffc22c049e2b4881ca8d3a1a;p=idzebra-moved-to-github.git diff --git a/doc/introduction.xml b/doc/introduction.xml index 29d759d..47096a3 100644 --- a/doc/introduction.xml +++ b/doc/introduction.xml @@ -1,35 +1,68 @@ - + Introduction
Overview + + &zebra; is a free, fast, friendly information management system. It can + index records in &xml;/&sgml;, &marc;, e-mail archives and many other + formats, and quickly find them using a combination of boolean + searching and relevance ranking. Search-and-retrieve applications can + be written using &api;s in a wide variety of languages, communicating + with the &zebra; server using industry-standard information-retrieval + protocols or web services. + + + &zebra; is licensed Open Source, and can be + deployed by anyone for any purpose without license fees. The C source + code is open to anybody to read and change under the GPL license. + + + &zebra; is a networked component which acts as a reliable &z3950; server + for both record/document search, presentation, insert, update and + delete operations. In addition, it understands the &sru; family of + webservices, which exist in &rest; &get;/&post; and truly &soap; flavors. + + + &zebra; is available as MS Windows 2003 Server (32 bit) self-extracting + package as well as GNU/Debian Linux (32 bit and 64 bit) precompiled + packages. It has been deployed successfully on other Unix systems, + including Sun Sparc, HP Unix, and many variants of Linux and BSD + based systems. + + + http://www.indexdata.com/zebra/ + http://ftp.indexdata.dk/pub/zebra/win32/ + http://ftp.indexdata.dk/pub/zebra/debian/ + + - Zebra + &zebra; is a high-performance, general-purpose structured text indexing and retrieval engine. It reads records in a - variety of input formats (eg. email, XML, MARC) and provides access + variety of input formats (eg. email, &xml;, &marc;) and provides access to them through a powerful combination of boolean search expressions and relevance-ranked free-text queries. - Zebra supports large databases (tens of millions of records, + &zebra; supports large databases (tens of millions of records, tens of gigabytes of data). It allows safe, incremental - database updates on live systems. Because Zebra supports - the industry-standard information retrieval protocol, Z39.50, - you can search Zebra databases using an enormous variety of + database updates on live systems. Because &zebra; supports + the industry-standard information retrieval protocol, &z3950;, + you can search &zebra; databases using an enormous variety of programs and toolkits, both commercial and free, which understand this protocol. Application libraries are available to allow bespoke clients to be written in Perl, C, C++, Java, Tcl, Visual - Basic, Python, PHP and more - see the - ZOOM web site + Basic, Python, &php; and more - see the + &zoom; web site for more information on some of these client toolkits. - This document is an introduction to the Zebra system. It explains + This document is an introduction to the &zebra; system. It explains how to compile the software, how to prepare your first database, and how to configure the server to give you the functionality that you need. @@ -37,11 +70,11 @@
- Zebra Features Overview + &zebra; Features Overview - Zebra Features Overview + &zebra; Features Overview @@ -54,24 +87,24 @@ Boolean query language - CQL and RPN/PQF - The type-1 Reverse Polish Notation (RPN) - and it's textual representation Prefix Query Format (PQF) are - supported. The Common Query Language (CQL) can be configured as - a mapping from CQL to RPN/PQF + &cql; and &rpn;/&pqf; + The type-1 Reverse Polish Notation (&rpn;) + and it's textual representation Prefix Query Format (&pqf;) are + supported. The Common Query Language (&cql;) can be configured as + a mapping from &cql; to &rpn;/&pqf; Operation types - Z39.50/SRU explain, search, and scan + &z3950;/&sru; explain, search, and scan Recursive boolean query tree - CQL and RPN/PQF - Both CQL and RPN/PQF allow atomic query parts (APT) to + &cql; and &rpn;/&pqf; + Both &cql; and &rpn;/&pqf; allow atomic query parts (&apt;) to be combined into complex boolean query trees @@ -86,8 +119,8 @@ Complex semi-structured Documents - XML and GRS-1 Documents - Both XML and GRS-1 documents exhibit a DOM like internal + &xml; and &grs1; Documents + Both &xml; and &grs1; documents exhibit a &dom; like internal representation allowing for complex indexing and display rules @@ -105,12 +138,12 @@ Input document formats - XML, SGML, Text, ISO2709 (MARC) + &xml;, &sgml;, Text, ISO2709 (&marc;) A system of input filters driven by regular expressions allows most ASCII-based data formats to be easily processed. - SGML, XML, ISO2709 (MARC), and raw text are also + &sgml;, &xml;, ISO2709 (&marc;), and raw text are also supported. @@ -125,7 +158,7 @@ Document storageIndex-only, Key storage, Document storageData can be, and usually is, imported - into Zebra's own storage, but Zebra can also refer to + into &zebra;'s own storage, but &zebra; can also refer to external files, building and maintaining indexes of "live" collections. @@ -145,26 +178,26 @@ Remote update - Z39.50 extended services + &z3950; extended services Supported Platforms UNIX, Linux, Windows (NT/2000/2003/XP) - Zebra is written in portable C, so it runs on most + &zebra; is written in portable C, so it runs on most Unix-like systems as well as Windows (NT/2000/2003/XP). Binary distributions are available for GNU/Debian Linux and Windows - Z39.50 - Z39.50 protocol support + &z3950; + &z3950; protocol support Protocol facilities: Init, Search, Present (retrieval), Segmentation (support for very large records), Delete, Scan (index browsing), Sort, Close and support for the ``update'' - Extended Service to add or replace an existing XML + Extended Service to add or replace an existing &xml; record. Piggy-backed presents are honored in the search request. Named result sets are supported. @@ -173,18 +206,18 @@ Record Syntaxes Multiple record syntaxes - for data retrieval: GRS-1, SUTRS, - XML, ISO2709 (MARC), etc. Records can be mapped between record syntaxes + for data retrieval: &grs1;, &sutrs;, + &xml;, ISO2709 (&marc;), etc. Records can be mapped between record syntaxes and schemas on the fly. Web Service support - SRU GET/POST/SOAP + &sru_gps; The protocol operations explain, searchRetrieve and scan - are supported. CQL to internal - query model RPN conversion is supported. Extended RPN queries + are supported. &cql; to internal + query model &rpn; conversion is supported. Extended RPN queries for search/retrieve and scan are supported. @@ -251,9 +284,9 @@
- References and Zebra based Applications + References and &zebra; based Applications - Zebra has been deployed in numerous applications, in both the + &zebra; has been deployed in numerous applications, in both the academic and commercial worlds, in application domains as diverse as bibliographic catalogues, geospatial information, structured vocabulary browsing, government information locators, civic @@ -278,19 +311,19 @@ LibLime, a company that is marketing and supporting Koha, adds in - the new release of Koha 3.0 the Zebra + the new release of Koha 3.0 the &zebra; database server to drive its bibliographic database. In early 2005, the Koha project development team began looking at - ways to improve MARC support and overcome scalability limitations + ways to improve &marc; support and overcome scalability limitations in the Koha 2.x series. After extensive evaluations of the best of the Open Source textual database engines - including MySQL full-text searching, PostgreSQL, Lucene and Plucene - the team - selected Zebra. + selected &zebra;. - "Zebra completely eliminates scalability limitations, because it + "&zebra; completely eliminates scalability limitations, because it can support tens of millions of records." explained Joshua Ferraro, LibLime's Technology President and Koha's Project Release Manager. "Our performance tests showed search results in @@ -298,16 +331,16 @@ modest i386 900Mhz test server." - "Zebra also includes support for true boolean search expressions + "&zebra; also includes support for true boolean search expressions and relevance-ranked free-text queries, both of which the Koha - 2.x series lack. Zebra also supports incremental and safe + 2.x series lack. &zebra; also supports incremental and safe database updates, which allow on-the-fly record - management. Finally, since Zebra has at its heart the Z39.50 + management. Finally, since &zebra; has at its heart the &z3950; protocol, it greatly improves Koha's support for that critical library standard." - Although the bibliographic database will be moved to Zebra, Koha + Although the bibliographic database will be moved to &zebra;, Koha 3.0 will continue to use a relational SQL-based database design for the 'factual' database. "Relational database managers have their strengths, in spite of their inability to handle large @@ -332,13 +365,13 @@ from virtually any computer with an Internet connection, has template based layout allowing anyone to alter the visual appearance of Emilda, and is - XML based language for fast and easy portability to virtually any + &xml; based language for fast and easy portability to virtually any language. Currently, Emilda is used at three schools in Espoo, Finland. - As a surplus, 100% MARC compatibility has been achieved using the - Zebra Server from Index Data as backend server. + As a surplus, 100% &marc; compatibility has been achieved using the + &zebra; Server from Index Data as backend server.
@@ -349,18 +382,18 @@ is a netbased library service offering all traditional functions on a very high level plus many new services. Reindex.net is a comprehensive and powerful WEB system - based on standards such as XML and Z39.50. - updates. Reindex supports MARC21, danMARC eller Dublin Core with + based on standards such as &xml; and &z3950;. + updates. Reindex supports &marc21;, dan&marc; eller Dublin Core with UTF8-encoding. - Reindex.net runs on GNU/Debian Linux with Zebra and Simpleserver + Reindex.net runs on GNU/Debian Linux with &zebra; and Simpleserver from Index Data for bibliographic data. The relational database system - Sybase 9 XML is used for + Sybase 9 &xml; is used for administrative data. - Internally MARCXML is used for bibliographical records. Update - utilizes Z39.50 extended services. + Internally &marcxml; is used for bibliographical records. Update + utilizes &z3950; extended services. @@ -422,11 +455,11 @@ bioinformatics. - The Zebra information retrieval indexing machine is used inside + The &zebra; information retrieval indexing machine is used inside the Alvis framework to manage huge collections of natural language processed and - enhanced XML data, coming from a topic relevant web crawl. - In this application, Zebra swallows and manages 37GB of XML data + enhanced &xml; data, coming from a topic relevant web crawl. + In this application, &zebra; swallows and manages 37GB of &xml; data in about 4 hours, resulting in search times of fractions of seconds. @@ -448,10 +481,10 @@ The member libraries send in data files representing their periodicals, including both brief bibliographic data and summary - holdings. Then 21 individual Z39.50 targets are created, each - using Zebra, and all mounted on the single hardware server. - The live service provides a web gateway allowing Z39.50 searching - of all of the targets or a selection of them. Zebra's small + holdings. Then 21 individual &z3950; targets are created, each + using &zebra;, and all mounted on the single hardware server. + The live service provides a web gateway allowing &z3950; searching + of all of the targets or a selection of them. &zebra;'s small footprint allows a relatively modest system to comfortably host the 21 servers. @@ -462,17 +495,17 @@
- NLI-Z39.50 - a Natural Language Interface for Libraries + NLI-&z3950; - a Natural Language Interface for Libraries Fernuniversität Hagen in Germany have developed a natural language interface for access to library databases. In order to evaluate this interface for recall and precision, they - chose Zebra as the basis for retrieval effectiveness. The Zebra + chose &zebra; as the basis for retrieval effectiveness. The &zebra; server contains a copy of the GIRT database, consisting of more - than 76000 records in SGML format (bibliographic records from - social science), which are mapped to MARC for presentation. + than 76000 records in &sgml; format (bibliographic records from + social science), which are mapped to &marc; for presentation. (GIRT is the German Indexing and Retrieval Testdatabase. It is a @@ -494,7 +527,7 @@
Various web indexes - Zebra has been used by a variety of institutions to construct + &zebra; has been used by a variety of institutions to construct indexes of large web sites, typically in the region of tens of millions of pages. In this role, it functions somewhat similarly to the engine of google or altavista, but for a selected intranet @@ -504,7 +537,7 @@ For example, Liverpool University's web-search facility (see on the home page at - and many sub-pages) works by relevance-searching a Zebra database + and many sub-pages) works by relevance-searching a &zebra; database which is populated by the Harvest-NG web-crawling software. @@ -514,31 +547,31 @@ Kang-Jin Lee - has recently modified the Harvest web indexer to use Zebra as + has recently modified the Harvest web indexer to use &zebra; as its native repository engine. His comments on the switch over from the old engine are revealing:
- The first results after some testing with Zebra are very + The first results after some testing with &zebra; are very promising. The tests were done with around 220,000 SOIF files, which occupies 1.6GB of disk space. - Building the index from scratch takes around one hour with Zebra + Building the index from scratch takes around one hour with &zebra; where [old-engine] needs around five hours. While [old-engine] - blocks search requests when updating its index, Zebra can still + blocks search requests when updating its index, &zebra; can still answer search requests. [...] - Zebra supports incremental indexing which will speed up indexing + &zebra; supports incremental indexing which will speed up indexing even further. While the search time of [old-engine] varies from some seconds - to some minutes depending how expensive the query is, Zebra + to some minutes depending how expensive the query is, &zebra; usually takes around one to three seconds, even for expensive queries. [...] - Zebra can search more than 100 times faster than [old-engine] + &zebra; can search more than 100 times faster than [old-engine] and can process multiple search requests simultaneously @@ -553,20 +586,20 @@
Support - You can get support for Zebra from at least three sources. + You can get support for &zebra; from at least three sources. - First, there's the Zebra web site at + First, there's the &zebra; web site at , which always has the most recent version available for download. - If you have a problem with Zebra, the first thing to do is see + If you have a problem with &zebra;, the first thing to do is see whether it's fixed in the current release. - Second, there's the Zebra mailing list. Its home page at + Second, there's the &zebra; mailing list. Its home page at includes a complete archive of all messages that have ever been - posted on the list. The Zebra mailing list is used both for + posted on the list. The &zebra; mailing list is used both for announcements from the authors (new releases, bug fixes, etc.) and general discussion. You are welcome to seek support there. Join by filling the form on the list home page. @@ -594,29 +627,29 @@ - Improved support for XML in search and retrieval. Eventually, - the goal is for Zebra to pull double duty as a flexible - information retrieval engine and high-performance XML + Improved support for &xml; in search and retrieval. Eventually, + the goal is for &zebra; to pull double duty as a flexible + information retrieval engine and high-performance &xml; repository. The recent addition of XPath searching is one example of the kind of enhancement we're working on. - There is also the experimental ALVIS XSLT - XML input filter, which unleashes the full power of DOM based - XSLT transformations during indexing and record retrieval. Work + There is also the experimental ALVIS &xslt; + &xml; input filter, which unleashes the full power of &dom; based + &xslt; transformations during indexing and record retrieval. Work on this filter has been sponsored by the ALVIS EU project . We expect this filter to mature soon, as it is planned to be included in the version 2.0 - release of Zebra. + release of &zebra;. - Finalisation and documentation of Zebra's C programming - API, allowing updates, database management and other functions - not readily expressed in Z39.50. We will also consider - exposing the API through SOAP. + Finalisation and documentation of &zebra;'s C programming + &api;, allowing updates, database management and other functions + not readily expressed in &z3950;. We will also consider + exposing the &api; through &soap;.