X-Git-Url: http://git.indexdata.com/?p=idzebra-moved-to-github.git;a=blobdiff_plain;f=doc%2Farchitecture.xml;h=5b3a27ebd2b5cc89cc08f7225dca9864c90487be;hp=cecd978e9267944be0d5b32d498dcd145109de9c;hb=31b0ba1ecb737f9db4cf3340e93964c6679897fd;hpb=b00064c85119fb3a6ca07f809f41d8f97f192165 diff --git a/doc/architecture.xml b/doc/architecture.xml index cecd978..5b3a27e 100644 --- a/doc/architecture.xml +++ b/doc/architecture.xml @@ -1,5 +1,4 @@ - Overview of &zebra; Architecture
@@ -87,9 +86,9 @@ Search Evaluation - by execution of search requests expressed in &pqf;/&rpn; + by execution of search requests expressed in &acro.pqf;/&acro.rpn; data structures, which are handed over from - the &yaz; server frontend &api;. Search evaluation includes + the &yaz; server frontend &acro.api;. Search evaluation includes construction of hit lists according to boolean combinations of simpler searches. Fast performance is achieved by careful use of index structures, and by evaluation specific index hit @@ -112,7 +111,7 @@ Record Presentation returns - possibly ranked - result sets, hit - numbers, and the like internal data to the &yaz; server backend &api; + numbers, and the like internal data to the &yaz; server backend &acro.api; for shipping to the client. Each individual filter module implements it's own specific presentation formats. @@ -149,7 +148,7 @@
&zebra; Searcher/Retriever - This is the executable which runs the &z3950;/&sru;/&srw; server and + This is the executable which runs the &acro.z3950;/&acro.sru;/&acro.srw; server and glues together the core libraries and the filter modules to one great Information Retrieval server application. @@ -163,21 +162,21 @@ &yaz; Server Frontend The &yaz; server frontend is - a full fledged stateful &z3950; server taking client + a full fledged stateful &acro.z3950; server taking client connections, and forwarding search and scan requests to the &zebra; core indexer. - In addition to &z3950; requests, the &yaz; server frontend acts + In addition to &acro.z3950; requests, the &yaz; server frontend acts as HTTP server, honoring - &sru; &soap; + &acro.sru; &acro.soap; requests, and - &sru; &rest; + &acro.sru; &acro.rest; requests. Moreover, it can translate incoming - &cql; + &acro.cql; queries to - &pqf; + &acro.pqf; queries, if correctly configured. @@ -185,7 +184,7 @@ &yaz; is an Open Source toolkit that allows you to develop software using the - &ansi; &z3950;/ISO23950 standard for information retrieval. + &acro.ansi; &acro.z3950;/ISO23950 standard for information retrieval. It is packaged in the Debian packages yaz and libyaz. @@ -208,21 +207,21 @@
- &dom; &xml; Record Model and Filter Module + &acro.dom; &acro.xml; Record Model and Filter Module - The &dom; &xml; filter uses a standard &dom; &xml; structure as + The &acro.dom; &acro.xml; filter uses a standard &acro.dom; &acro.xml; structure as internal data model, and can thus parse, index, and display - any &xml; document. + any &acro.xml; document. - A parser for binary &marc; records based on the ISO2709 library + A parser for binary &acro.marc; records based on the ISO2709 library standard is provided, it transforms these to the internal - &marcxml; &dom; representation. + &acro.marcxml; &acro.dom; representation. - The internal &dom; &xml; representation can be fed into four - different pipelines, consisting of arbitraily many sucessive - &xslt; transformations; these are for + The internal &acro.dom; &acro.xml; representation can be fed into four + different pipelines, consisting of arbitrarily many successive + &acro.xslt; transformations; these are for input parsing and initial transformations, @@ -235,55 +234,55 @@ - The &dom; &xml; filter pipelines use &xslt; (and if supported on - your platform, even &exslt;), it brings thus full &xpath; + The &acro.dom; &acro.xml; filter pipelines use &acro.xslt; (and if supported on + your platform, even &acro.exslt;), it brings thus full &acro.xpath; support to the indexing, storage and display rules of not only - &xml; documents, but also binary &marc; records. + &acro.xml; documents, but also binary &acro.marc; records. - Finally, the &dom; &xml; filter allows for static ranking at index + Finally, the &acro.dom; &acro.xml; filter allows for static ranking at index time, and to to sort hit lists according to predefined static ranks. - Details on the experimental &dom; &xml; filter are found in + Details on the experimental &acro.dom; &acro.xml; filter are found in . The Debian package libidzebra-2.0-mod-dom - contains the &dom; filter module. + contains the &acro.dom; filter module.
- ALVIS &xml; Record Model and Filter Module + ALVIS &acro.xml; Record Model and Filter Module The functionality of this record model has been improved and - replaced by the &dom; &xml; record model. See + replaced by the &acro.dom; &acro.xml; record model. See . - The Alvis filter for &xml; files is an &xslt; based input + The Alvis filter for &acro.xml; files is an &acro.xslt; based input filter. - It indexes element and attribute content of any thinkable &xml; format - using full &xpath; support, a feature which the standard &zebra; - &grs1; &sgml; and &xml; filters lacked. The indexed documents are - parsed into a standard &xml; &dom; tree, which restricts record size + It indexes element and attribute content of any thinkable &acro.xml; format + using full &acro.xpath; support, a feature which the standard &zebra; + &acro.grs1; &acro.sgml; and &acro.xml; filters lacked. The indexed documents are + parsed into a standard &acro.xml; &acro.dom; tree, which restricts record size according to availability of memory. The Alvis filter - uses &xslt; display stylesheets, which let + uses &acro.xslt; display stylesheets, which let the &zebra; DB administrator associate multiple, different views on - the same &xml; document type. These views are chosen on-the-fly in + the same &acro.xml; document type. These views are chosen on-the-fly in search time. In addition, the Alvis filter configuration is not bound to the - arcane &bib1; &z3950; library catalogue indexing traditions and + arcane &acro.bib1; &acro.z3950; library catalogue indexing traditions and folklore, and is therefore easier to understand. @@ -292,11 +291,11 @@ static ranks. This imposes no overhead at all, both search and indexing perform still O(1) irrespectively of document - collection size. This feature resembles Googles pre-ranking using - their Pagerank algorithm. + collection size. This feature resembles Google's pre-ranking using + their PageRank algorithm. - Details on the experimental Alvis &xslt; filter are found in + Details on the experimental Alvis &acro.xslt; filter are found in . @@ -306,34 +305,34 @@
- &grs1; Record Model and Filter Modules + &acro.grs1; Record Model and Filter Modules The functionality of this record model has been improved and - replaced by the &dom; &xml; record model. See + replaced by the &acro.dom; &acro.xml; record model. See . - The &grs1; filter modules described in + The &acro.grs1; filter modules described in - are all based on the &z3950; specifications, and it is absolutely - mandatory to have the reference pages on &bib1; attribute sets on - you hand when configuring &grs1; filters. The GRS filters come in + are all based on the &acro.z3950; specifications, and it is absolutely + mandatory to have the reference pages on &acro.bib1; attribute sets on + you hand when configuring &acro.grs1; filters. The GRS filters come in different flavors, and a short introduction is needed here. - &grs1; filters of various kind have also been called ABS filters due + &acro.grs1; filters of various kind have also been called ABS filters due to the *.abs configuration file suffix. The grs.marc and grs.marcxml filters are suited to parse and - index binary and &xml; versions of traditional library &marc; records + index binary and &acro.xml; versions of traditional library &acro.marc; records based on the ISO2709 standard. The Debian package for both filters is libidzebra-2.0-mod-grs-marc. - &grs1; TCL scriptable filters for extensive user configuration come + &acro.grs1; TCL scriptable filters for extensive user configuration come in two flavors: a regular expression filter grs.regx using TCL regular expressions, and a general scriptable TCL filter called @@ -342,7 +341,7 @@ libidzebra-2.0-mod-grs-regx Debian package. - A general purpose &sgml; filter is called + A general purpose &acro.sgml; filter is called grs.sgml. This filter is not yet packaged, but planned to be in the libidzebra-2.0-mod-grs-sgml Debian package. @@ -352,8 +351,8 @@ libidzebra-2.0-mod-grs-xml includes the grs.xml filter which uses Expat to - parse records in &xml; and turn them into ID&zebra;'s internal &grs1; node - trees. Have also a look at the Alvis &xml;/&xslt; filter described in + parse records in &acro.xml; and turn them into ID&zebra;'s internal &acro.grs1; node + trees. Have also a look at the Alvis &acro.xml;/&acro.xslt; filter described in the next session.
@@ -394,8 +393,8 @@ When records are accessed by the system, they are represented - in their local, or native format. This might be &sgml; or HTML files, - News or Mail archives, &marc; records. If the system doesn't already + in their local, or native format. This might be &acro.sgml; or HTML files, + News or Mail archives, &acro.marc; records. If the system doesn't already know how to read the type of data you need to store, you can set up an input filter by preparing conversion rules based on regular expressions and possibly augmented by a flexible scripting language @@ -422,7 +421,7 @@ Before transmitting records to the client, they are first converted from the internal structure to a form suitable for exchange - over the network - according to the &z3950; standard. + over the network - according to the &acro.z3950; standard. @@ -443,8 +442,8 @@ &zebra;'s internal index structure/data for a record. In particular, the regular record filters are not invoked when these are in use. - This can in some cases make the retrival faster than regular - retrieval operations (for &marc;, &xml; etc). + This can in some cases make the retrieval faster than regular + retrieval operations (for &acro.marc;, &acro.xml; etc). Special Retrieval Elements @@ -460,7 +459,7 @@ zebra::meta::sysno Get &zebra; record system ID - &xml; and &sutrs; + &acro.xml; and &acro.sutrs; zebra::data @@ -470,12 +469,12 @@ zebra::meta Get &zebra; record internal metadata - &xml; and &sutrs; + &acro.xml; and &acro.sutrs; zebra::index Get all indexed keys for record - &xml; and &sutrs; + &acro.xml; and &acro.sutrs; @@ -484,7 +483,7 @@ Get indexed keys for field f for record - &xml; and &sutrs; + &acro.xml; and &acro.sutrs; @@ -494,7 +493,35 @@ Get indexed keys for field f and type t for record - &xml; and &sutrs; + &acro.xml; and &acro.sutrs; + + + + zebra::snippet + + + Get snippet for record for one or more indexes (f1,f2,..). + This includes a phrase from the original + record at the point where a match occurs (for a query). By default + give terms before - and after are included in the snippet. The + matching terms are enclosed within element + <s>. The snippet facility requires + Zebra 2.0.16 or later. + + &acro.xml; and &acro.sutrs; + + + + zebra::facet::f1:t1,f2:t2,.. + + + Get facet of a result set. The facet result is returned + as if it was a normal record, while in reality is a + recap of most "important" terms in a result set for the fields + given. + The facet facility first appeared in Zebra 2.0.20. + + &acro.xml; @@ -529,7 +556,7 @@ Z> elements zebra::meta::sysno Z> s 1+1 - displays in &xml; record syntax only internal + displays in &acro.xml; record syntax only internal record system number, whereas Z> f @attr 1=title my @@ -537,7 +564,7 @@ Z> elements zebra::meta Z> s 1+1 - displays all available metadata on the record. These include sytem + displays all available metadata on the record. These include system number, database name, indexed filename, filter used for indexing, score and static ranking information and finally bytesize of record. @@ -560,7 +587,7 @@ Z> s 1+1 will display all indexed tokens from all indexed fields of the - first record, and it will display in &sutrs; + first record, and it will display in &acro.sutrs; record syntax, whereas Z> f @attr 1=title my @@ -570,13 +597,13 @@ Z> elements zebra::index::title:p Z> s 1+1 - displays in &xml; record syntax only the content + displays in &acro.xml; record syntax only the content of the zebra string index title, or even only the type p phrase indexed part of it. - Trying to access numeric &bib1; use + Trying to access numeric &acro.bib1; use attributes or trying to access non-existent zebra intern string access points will result in a Diagnostic 25: Specified element set 'name not valid for specified database.