X-Git-Url: http://git.indexdata.com/?p=idzebra-moved-to-github.git;a=blobdiff_plain;f=doc%2Frecordmodel-domxml.xml;h=16d7001e69f849232f0253a9f8285be40b51868b;hp=8dfcdb6cfb49102c770f5752eb80d7b6015a2724;hb=c3ff843e467932c6027a8b3b2ebda7b44612447e;hpb=799d116b6a50e44e9eeccf17ed81ffb70220b4f4 diff --git a/doc/recordmodel-domxml.xml b/doc/recordmodel-domxml.xml index 8dfcdb6..16d7001 100644 --- a/doc/recordmodel-domxml.xml +++ b/doc/recordmodel-domxml.xml @@ -1,186 +1,185 @@ - - - &dom; &xml; Record Model and Filter Module + + &acro.dom; &acro.xml; Record Model and Filter Module The record model described in this chapter applies to the fundamental, - structured &xml; - record type dom, introduced in - . The &dom; &xml; record model - is experimental, and it's inner workings might change in future + structured &acro.xml; + record type &acro.dom;, introduced in + . The &acro.dom; &acro.xml; record model + is experimental, and its inner workings might change in future releases of the &zebra; Information Server. - - + +
- &dom; Record Filter Architecture + &acro.dom; Record Filter Architecture - - The &dom; &xml; filter uses a standard &dom; &xml; structure as - internal data model, and can therefore parse, index, and display - any &xml; document type. It is wellsuited to work on - standardized &xml;-based formats such as Dublin Core, MODS, METS, - MARCXML, OAI-PMH, RSS, and performs equally well on any other - non-standard &xml; format. - - - A parser for binary &marc; records based on the ISO2709 library - standard is provided, it transforms these to the internal - &marcxml; &dom; representation. Other binary document parsers - are planned to follow. - + + The &acro.dom; &acro.xml; filter uses a standard &acro.dom; &acro.xml; structure as + internal data model, and can therefore parse, index, and display + any &acro.xml; document type. It is well suited to work on + standardized &acro.xml;-based formats such as Dublin Core, MODS, METS, + MARCXML, OAI-PMH, RSS, and performs equally well on any other + non-standard &acro.xml; format. + + + A parser for binary &acro.marc; records based on the ISO2709 library + standard is provided, it transforms these to the internal + &acro.marcxml; &acro.dom; representation. Other binary document parsers + are planned to follow. + - - The &dom; filter architecture consists of four - different pipelines, each being a chain of arbitraily many sucessive - &xslt; transformations of the internal &dom; &xml; - representations of documents. - + + The &acro.dom; filter architecture consists of four + different pipelines, each being a chain of arbitrarily many successive + &acro.xslt; transformations of the internal &acro.dom; &acro.xml; + representations of documents. + -
- &dom; &xml; filter architecture - - - - - - - - - - - [Here there should be a diagram showing the &dom; &xml; - filter architecture, but is seems that your - tool chain has not been able to include the diagram in this - document.] - - - -
- - - - &dom; &xml; filter pipelines overview - - - - Name - When - Description - Input - Output - - - - - - input - first - input parsing and initial - transformations to common &xml; format - Input raw &xml; record buffers, &xml; streams and - binary &marc; buffers - Common &xml; &dom; - - - extract - second - indexing term extraction - transformations - Common &xml; &dom; - Indexing &xml; &dom; - - - store - second - transformations before internal document - storage - Common &xml; &dom; - Storage &xml; &dom; - - - retrieve - third - multiple document retrieve transformations from - storage to different output - formats are possible - Storage &xml; &dom; - Output &xml; syntax in requested formats - - - -
+
+ &acro.dom; &acro.xml; filter architecture + + + + + + + + + + + [Here there should be a diagram showing the &acro.dom; &acro.xml; + filter architecture, but is seems that your + tool chain has not been able to include the diagram in this + document.] + + + +
+ + + + &acro.dom; &acro.xml; filter pipelines overview + + + + Name + When + Description + Input + Output + + + + + + input + first + input parsing and initial + transformations to common &acro.xml; format + Input raw &acro.xml; record buffers, &acro.xml; streams and + binary &acro.marc; buffers + Common &acro.xml; &acro.dom; + + + extract + second + indexing term extraction + transformations + Common &acro.xml; &acro.dom; + Indexing &acro.xml; &acro.dom; + + + store + second + transformations before internal document + storage + Common &acro.xml; &acro.dom; + Storage &acro.xml; &acro.dom; + + + retrieve + third + multiple document retrieve transformations from + storage to different output + formats are possible + Storage &acro.xml; &acro.dom; + Output &acro.xml; syntax in requested formats + + + +
- - The &dom; &xml; filter pipelines use &xslt; (and if supported on - your platform, even &exslt;), it brings thus full &xpath; - support to the indexing, storage and display rules of not only - &xml; documents, but also binary &marc; records. - -
+ + The &acro.dom; &acro.xml; filter pipelines use &acro.xslt; (and if supported on + your platform, even &acro.exslt;), it brings thus full &acro.xpath; + support to the indexing, storage and display rules of not only + &acro.xml; documents, but also binary &acro.marc; records. + + -
- &dom; &xml; filter pipeline configuration +
+ &acro.dom; &acro.xml; filter pipeline configuration - The experimental, loadable &dom; &xml;/&xslt; filter module - mod-dom.so + The experimental, loadable &acro.dom; &acro.xml;/&acro.xslt; filter module + mod-dom.so is invoked by the zebra.cfg configuration statement recordtype.xml: dom.db/filter_dom_conf.xml - In this example the &dom; &xml; filter is configured to work - on all data files with suffix + In this example the &acro.dom; &acro.xml; filter is configured to work + on all data files with suffix *.xml, where the configuration file is found in the path db/filter_dom_conf.xml. - The &dom; &xslt; filter configuration file must be - valid &xml;. It might look like this: - - - - - - - - - - - - - - - - - - - + The &acro.dom; &acro.xslt; filter configuration file must be + valid &acro.xml;. It might look like this: + + + + + + + + + + + + + + + + + + + - ]]> + ]]> - The root &xml; element <dom> and all other &dom; - &xml; filter elements are residing in the namespace - http://indexdata.com/zebra-2.0. + The root &acro.xml; element <dom> and all other &acro.dom; + &acro.xml; filter elements are residing in the namespace + xmlns="http://indexdata.com/zebra-2.0". All pipeline definition elements - i.e. the - <input>, - <extact>, - <store>, and - <retrieve> elements - are optional. - Missing pipeline definitions are just interpreted - do-nothing identity pipelines. + <input>, + <extract>, + <store>, and + <retrieve> elements - are optional. + Missing pipeline definitions are just interpreted + do-nothing identity pipelines. - All pipeine definition elements may contain zero or more + All pipeline definition elements may contain zero or more ]]> - &xslt; transformation instructions, which are performed + &acro.xslt; transformation instructions, which are performed sequentially from top to bottom. The paths in the stylesheet attributes are relative to zebras working directory, or absolute to the file @@ -189,493 +188,745 @@
- Input pipeline - - The <input> pipeline definition element - may contain either one &xml; Reader definition - ]]>, used to split - an &xml; collection input stream into individual &xml; &dom; - documents at the prescribed element level, - or one &marc; binary - parsing instruction - ]]>, which defines - a conversion to &marcxml; format &dom; trees. The allowed values - of the inputcharset attribute depend on your - local iconv set-up. - - - Both input parsers deliver individual &dom; &xml; documents to the - following chain of zero or more - ]]> - &xslt; transformations. At the end of this pipeline, the documents - are in the common format, used to feed both the - <extact> and + Input pipeline + + The <input> pipeline definition element + may contain either one &acro.xml; Reader definition + ]]>, used to split + an &acro.xml; collection input stream into individual &acro.xml; &acro.dom; + documents at the prescribed element level, + or one &acro.marc; binary + parsing instruction + ]]>, which defines + a conversion to &acro.marcxml; format &acro.dom; trees. The allowed values + of the inputcharset attribute depend on your + local iconv set-up. + + + Both input parsers deliver individual &acro.dom; &acro.xml; documents to the + following chain of zero or more + ]]> + &acro.xslt; transformations. At the end of this pipeline, the documents + are in the common format, used to feed both the + <extract> and <store> pipelines. - +
- Extract pipeline + Extract pipeline + + The <extract> pipeline takes documents + from any common &acro.dom; &acro.xml; format to the &zebra; specific + indexing &acro.dom; &acro.xml; format. + It may consist of zero ore more + ]]> + &acro.xslt; transformations, and the outcome is handled to the + &zebra; core to drive the process of building the inverted + indexes. See + for + details. +
- Store pipeline + Store pipeline + The <store> pipeline takes documents + from any common &acro.dom; &acro.xml; format to the &zebra; specific + storage &acro.dom; &acro.xml; format. + It may consist of zero ore more + ]]> + &acro.xslt; transformations, and the outcome is handled to the + &zebra; core for deposition into the internal storage system.
- Retrieve pipeline - + Retrieve pipeline - All named stylesheets defined inside - schema element tags - are for presentation after search, including - the indexing stylesheet (which is a great debugging help). The - names defined in the name attributes must be - unique, these are the literal schema or - element set names used in - &srw;, - &sru; and - &z3950; protocol queries. - + Finally, there may be one or more + <retrieve> pipeline definitions, each + of them again consisting of zero or more + ]]> + &acro.xslt; transformations. These are used for document + presentation after search, and take the internal storage &acro.dom; + &acro.xml; to the requested output formats during record present + requests. + + + The possible multiple + <retrieve> pipeline definitions + are distinguished by their unique name + attributes, these are the literal schema or + element set names used in + &acro.srw;, + &acro.sru; and + &acro.z3950; protocol queries. +
-
- &dom; filter internal record representation - When indexing, an &xml; Reader is invoked to split the input - files into suitable record &xml; pieces. Each record piece is then - transformed to an &xml; &dom; structure, which is essentially the - record model. Only &xslt; transformations can be applied during - index, search and retrieval. Consequently, output formats are - restricted to whatever &xslt; can deliver from the record &xml; - structure, be it other &xml; formats, HTML, or plain text. In case - you have libxslt1 running with E&xslt; support, - you can use this functionality inside the &dom; - filter configuration &xslt; stylesheets. - -
+
+ Canonical Indexing Format -
- &dom; Canonical Indexing Format - The output of the indexing &xslt; stylesheets must contain - certain elements in the magic - xmlns:z="http://indexdata.dk/zebra/xslt/1" - namespace. The output of the &xslt; indexing transformation is then - parsed using &dom; methods, and the contained instructions are - performed on the magic elements and their - subtrees. - - For example, the output of the command - - xsltproc xsl/oai2index.xsl one-record.xml - - might look like this: - - <?xml version="1.0" encoding="UTF-8"?> - <z:record xmlns:z="http://indexdata.dk/zebra/xslt/1" - z:id="oai:JTRS:CP-3290---Volume-I" - z:rank="47896" - z:type="update"> - <z:index name="oai_identifier" type="0"> - oai:JTRS:CP-3290---Volume-I</z:index> - <z:index name="oai_datestamp" type="0">2004-07-09</z:index> - <z:index name="oai_setspec" type="0">jtrs</z:index> - <z:index name="dc_all" type="w"> - <z:index name="dc_title" type="w">Proceedings of the 4th - International Conference and Exhibition: - World Congress on Superconductivity - Volume I</z:index> - <z:index name="dc_creator" type="w">Kumar Krishen and *Calvin - Burnham, Editors</z:index> - </z:index> - </z:record> - - - This means the following: From the original &xml; file - one-record.xml (or from the &xml; record &dom; of the - same form coming from a splitted input file), the indexing - stylesheet produces an indexing &xml; record, which is defined by - the record element in the magic namespace - xmlns:z="http://indexdata.dk/zebra/xslt/1". - &zebra; uses the content of - z:id="oai:JTRS:CP-3290---Volume-I" as internal - record ID, and - in case static ranking is set - the content of - z:rank="47896" as static rank. Following the - discussion in - we see that this records is internally ordered - lexicographically according to the value of the string - oai:JTRS:CP-3290---Volume-I47896. - The type of action performed during indexing is defined by - z:type="update">, with recognized values - insert, update, and - delete. - - In this example, the following literal indexes are constructed: - - oai_identifier - oai_datestamp - oai_setspec - dc_all - dc_title - dc_creator - - where the indexing type is defined in the - type attribute - (any value from the standard configuration - file default.idx will do). Finally, any - text() node content recursively contained - inside the index will be filtered through the - appropriate charmap for character normalization, and will be - inserted in the index. - - - Specific to this example, we see that the single word - oai:JTRS:CP-3290---Volume-I will be literal, - byte for byte without any form of character normalization, - inserted into the index named oai:identifier, - the text - Kumar Krishen and *Calvin Burnham, Editors - will be inserted using the w character - normalization defined in default.idx into - the index dc:creator (that is, after character - normalization the index will keep the inidividual words - kumar, krishen, - and, calvin, - burnham, and editors), and - finally both the texts - Proceedings of the 4th International Conference and Exhibition: - World Congress on Superconductivity - Volume I - and - Kumar Krishen and *Calvin Burnham, Editors - will be inserted into the index dc:all using - the same character normalization map w. - - - Finally, this example configuration can be queried using &pqf; - queries, either transported by &z3950;, (here using a yaz-client) - - open localhost:9999 - Z> elem dc - Z> form xml - Z> - Z> f @attr 1=dc_creator Kumar - Z> scan @attr 1=dc_creator adam - Z> - Z> f @attr 1=dc_title @attr 4=2 "proceeding congress superconductivity" - Z> scan @attr 1=dc_title abc - ]]> - - or the proprietary - extentions x-pquery and - x-pScanClause to - &sru;, and &srw; - - - - See for more information on &sru;/&srw; - configuration, and or the &yaz; - &cql; section - for the details or the &yaz; frontend server. - - - Notice that there are no *.abs, - *.est, *.map, or other &grs1; - filter configuration files involves in this process, and that the - literal index names are used during search and retrieval. + &acro.dom; &acro.xml; indexing comes in two flavors: pure + processing-instruction governed plain &acro.xml; documents, and - very + similar to the Alvis filter indexing format - &acro.xml; documents + containing &acro.xml; <record> and + <index> instructions from the magic + namespace xmlns:z="http://indexdata.com/zebra-2.0". + +
+ Processing-instruction governed indexing format + + The output of the processing instruction driven + indexing &acro.xslt; stylesheets must contain + processing instructions named + zebra-2.0. + The output of the &acro.xslt; indexing transformation is then + parsed using &acro.dom; methods, and the contained instructions are + performed on the elements and their + subtrees directly following the processing instructions. + + + For example, the output of the command + + xsltproc dom-index-pi.xsl marc-one.xml + + might look like this: + + + + + + 11224466 + + How to program a computer + + ]]> + + +
+ +
+ Magic element governed indexing format + + The output of the indexing &acro.xslt; stylesheets must contain + certain elements in the magic + xmlns:z="http://indexdata.com/zebra-2.0" + namespace. The output of the &acro.xslt; indexing transformation is then + parsed using &acro.dom; methods, and the contained instructions are + performed on the magic elements and their + subtrees. + + + For example, the output of the command + + xsltproc dom-index-element.xsl marc-one.xml + + might look like this: + + + + 11224466 + + How to program a computer + + ]]> + + +
+ + +
+ Semantics of the indexing formats + + + Both indexing formats are defined with equal semantics and + behavior in mind: + + + &zebra; specific instructions are either + processing instructions named + zebra-2.0 or + elements contained in the namespace + xmlns:z="http://indexdata.com/zebra-2.0". + + + + There must be exactly one record + instruction, which sets the scope for the following, + possibly nested index instructions. + + + + + The unique record instruction + may have additional attributes id, + rank and type. + Attribute id is the value of the opaque ID + and may be any string not containing the whitespace character + ' '. + The rank attribute value must be a + non-negative integer. See + . + The type attribute specifies how the record + is to be treated. The following values may be given for + type: + + + insert + + + The record is inserted. If the record already exists, it is + skipped (i.e. not replaced). + + + + + replace + + + The record is replaced. If the record does not already exist, + it is skipped (i.e. not inserted). + + + + + delete + + + The record is deleted. If the record does not already exist, + a warning issued and rest of records are skipped in + from the input stream. + + + + + update + + + The record is inserted or replaced depending on whether the + record exists or not. This is the default behavior but may + be effectively changed by "outside" the scope of the DOM + filter by zebraidx commands or extended services updates. + + + + + adelete + + + The record is deleted. If the record does not already exist, + it is skipped (i.e. nothing is deleted). + + + + Requires version 2.0.54 or later. + + + + + + Note that the value of type is only used to + determine the action if and only if the Zebra indexer is running + in "update" mode (i.e zebraidx update) or if the specialUpdate + action of the + Extended + Service Update is used. + For this reason a specialUpdate may end up deleting records! + + + + Multiple and possible nested index + instructions must contain at least one + indexname:indextype + pair, and may contain multiple such pairs separated by the + whitespace character ' '. In each index + pair, the name and the type of the index is separated by a + colon character ':'. + + + + + Any index name consisting of ASCII letters, and following the + standard &zebra; rules will do, see + . + + + + + Index types are restricted to the values defined in + the standard configuration + file default.idx, see + and + for details. + + + + + &acro.dom; input documents which are not resulting in both one + unique valid + record instruction and one or more valid + index instructions can not be searched and + found. Therefore, + invalid document processing is aborted, and any content of + the <extract> and + <store> pipelines is discarded. + A warning is issued in the logs. + + + + + + The examples work as follows: + From the original &acro.xml; file + marc-one.xml (or from the &acro.xml; record &acro.dom; of the + same form coming from an <input> + pipeline), + the indexing + pipeline <extract> + produces an indexing &acro.xml; record, which is defined by + the record instruction + &zebra; uses the content of + z:id="11224466" + or + id=11224466 + as internal + record ID, and - in case static ranking is set - the content of + rank=42 + or + z:rank="42" + as static rank. + + + + In these examples, the following literal indexes are constructed: + + any:w + control:0 + title:w + title:p + title:s + + where the indexing type is defined after the + literal ':' character. + Any value from the standard configuration + file default.idx will do. + Finally, any + text() node content recursively contained + inside the <z:index> element, or any + element following a index processing instruction, + will be filtered through the + appropriate char map for character normalization, and will be + inserted in the named indexes. + + + Finally, this example configuration can be queried using &acro.pqf; + queries, either transported by &acro.z3950;, (here using a yaz-client) + + open localhost:9999 + Z> elem dc + Z> form xml + Z> + Z> find @attr 1=control @attr 4=3 11224466 + Z> scan @attr 1=control @attr 4=3 "" + Z> + Z> find @attr 1=title program + Z> scan @attr 1=title "" + Z> + Z> find @attr 1=title @attr 4=2 "How to program a computer" + Z> scan @attr 1=title @attr 4=2 "" + ]]> + + or the proprietary + extensions x-pquery and + x-pScanClause to + &acro.sru;, and &acro.srw; + + + + See for more information on &acro.sru;/&acro.srw; + configuration, and or the &yaz; + &acro.cql; section + for the details or the &yaz; frontend server. + + + Notice that there are no *.abs, + *.est, *.map, or other &acro.grs1; + filter configuration files involves in this process, and that the + literal index names are used during search and retrieval. + + + In case that we want to support the usual + bib-1 &acro.z3950; numeric access points, it is a + good idea to choose string index names defined in the default + configuration file tab/bib1.att, see + + + +
+
- &dom; Record Model Configuration + &acro.dom; Record Model Configuration -
- &dom; Indexing Configuration +
+ &acro.dom; Indexing Configuration - As mentioned above, there can be only one indexing - stylesheet, and configuration of the indexing process is a synonym - of writing an &xslt; stylesheet which produces &xml; output containing the - magic elements discussed in - . + As mentioned above, there can be only one indexing pipeline, + and configuration of the indexing process is a synonym + of writing an &acro.xslt; stylesheet which produces &acro.xml; output containing the + magic processing instructions or elements discussed in + . Obviously, there are million of different ways to accomplish this - task, and some comments and code snippets are in order to lead - our paduans on the right track to the good side of the force. + task, and some comments and code snippets are in order to + enlighten the wary. Stylesheets can be written in the pull or the push style: pull - means that the output &xml; structure is taken as starting point of - the internal structure of the &xslt; stylesheet, and portions of - the input &xml; are pulled out and inserted - into the right spots of the output &xml; structure. On the other - side, push &xslt; stylesheets are recursavly + means that the output &acro.xml; structure is taken as starting point of + the internal structure of the &acro.xslt; stylesheet, and portions of + the input &acro.xml; are pulled out and inserted + into the right spots of the output &acro.xml; structure. + On the other + side, push &acro.xslt; stylesheets are recursively calling their template definitions, a process which is commanded - by the input &xml; structure, and avake to produce some output &xml; - whenever some special conditions in the input styelsheets are + by the input &acro.xml; structure, and is triggered to produce + some output &acro.xml; + whenever some special conditions in the input stylesheets are met. The pull type is well-suited for input - &xml; with strong and well-defined structure and semantcs, like the - following &oai; indexing example, whereas the + &acro.xml; with strong and well-defined structure and semantics, like the + following &acro.oai; indexing example, whereas the push type might be the only possible way to - sort out deeply recursive input &xml; formats. + sort out deeply recursive input &acro.xml; formats. - + A pull stylesheet example used to index - &oai; harvested records could use some of the following template + &acro.oai; harvested records could use some of the following template definitions: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + xmlns:z="http://indexdata.com/zebra-2.0" + xmlns:oai="http://www.openarchives.org/&acro.oai;/2.0/" + xmlns:oai_dc="http://www.openarchives.org/&acro.oai;/2.0/oai_dc/" + xmlns:dc="http://purl.org/dc/elements/1.1/" + version="1.0"> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ]]> +
+ + +
+ &acro.dom; Indexing &acro.marcxml; - Notice also, - that the names and types of the indexes can be defined in the - indexing &xslt; stylesheet dynamically according to - content in the original &xml; records, which has - opportunities for great power and wizardery as well as grande - disaster. + The &acro.dom; filter allows indexing of both binary &acro.marc; records + and &acro.marcxml; records, depending on its configuration. + A typical &acro.marcxml; record might look like this: + + + 42 + 00366nam 22001698a 4500 + 11224466 + DLC + 00000000000000.0 + 910710c19910701nju 00010 eng + + 11224466 + + + DLC + DLC + + + 123-xyz + + + Jack Collins + + + How to program a computer + + + Penguin + + + 8710 + + + p. cm. + + + ]]> + + + + + It is easily possible to make string manipulation in the &acro.dom; + filter. For example, if you want to drop some leading articles + in the indexing of sort fields, you might want to pick out the + &acro.marcxml; indicator attributes to chop of leading substrings. If + the above &acro.xml; example would have an indicator + ind2="8" in the title field + 245, i.e. + + + How to program a computer + + ]]> + + one could write a template taking into account this information + to chop the first 8 characters from the + sorting index title:s like this: + + + + + 0 + + + + + + + + + + + + + + ]]> + + The output of the above &acro.marcxml; and &acro.xslt; excerpt would then be: + + How to program a computer + program a computer + ]]> + + and the record would be sorted in the title index under 'P', not 'H'. + +
+ + +
+ &acro.dom; Indexing Wizardry + + The names and types of the indexes can be defined in the + indexing &acro.xslt; stylesheet dynamically according to + content in the original &acro.xml; records, which has + opportunities for great power and wizardry as well as grande + disaster. The following excerpt of a push stylesheet - might - be a good idea according to your strict control of the &xml; - input format (due to rigerours checking against well-defined and - tight RelaxNG or &xml; Schema's, for example): + might + be a good idea according to your strict control of the &acro.xml; + input format (due to rigorous checking against well-defined and + tight RelaxNG or &acro.xml; Schema's, for example): - - - - + + + + + ]]> - This template creates indexes which have the name of the working - node of any input &xml; file, and assigns a '1' to the index. - The example query - find @attr 1=xyz 1 + This template creates indexes which have the name of the working + node of any input &acro.xml; file, and assigns a '1' to the index. + The example query + find @attr 1=xyz 1 finds all files which contain at least one - xyz &xml; element. In case you can not control + xyz &acro.xml; element. In case you can not control which element names the input files contain, you might ask for disaster and bad karma using this technique. One variation over the theme dynamically created - indexes will definitely be unwise: + indexes will definitely be unwise: - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + ]]> - Don't be tempted to cross - the line to the dark side of the force, paduan; this leads - to suffering and pain, and universal - disentigration of your project schedule. + Don't be tempted to play too smart tricks with the power of + &acro.xslt;, the above example will create zillions of + indexes with unpredictable names, resulting in severe &zebra; + index pollution.. -
+
-
- &dom; Exchange Formats - - An exchange format can be anything which can be the outcome of an - &xslt; transformation, as far as the stylesheet is registered in - the main &dom; &xslt; filter configuration file, see - . - In principle anything that can be expressed in &xml;, HTML, and - TEXT can be the output of a schema or - element set directive during search, as long as - the information comes from the - original input record &xml; &dom; tree - (and not the transformed and indexed &xml;!!). - +
+ Debuggig &acro.dom; Filter Configurations - In addition, internal administrative information from the &zebra; - indexer can be accessed during record retrieval. The following - example is a summary of the possibilities: + It can be very hard to debug a &acro.dom; filter setup due to the many + successive &acro.marc; syntax translations, &acro.xml; stream splitting and + &acro.xslt; transformations involved. As an aid, you have always the + power of the -s command line switch to the + zebraidz indexing command at your hand: - - - - - - - - - - - - - - - - - - - - - - ]]> + zebraidx -s -c zebra.cfg update some_record_stream.xml + This command line simulates indexing and dumps a lot of debug + information in the logs, telling exactly which transformations + have been applied, how the documents look like after each + transformation, and which record ids and terms are send to the indexer. +
-
- -
- &dom; Filter &oai; Indexing Example + - - - - - - +
- ---> + @@ -688,7 +939,7 @@ c) Main "dom" &xslt; filter config file: sgml-always-quote-attributes:t sgml-indent-step:1 sgml-indent-data:t - sgml-parent-document: "zebra.xml" + sgml-parent-document: "idzebra.xml" sgml-local-catalogs: nil sgml-namecase-general:t End: