-<chapter id="record-model-alvisxslt">
+ <chapter id="record-model-alvisxslt">
<title>ALVIS &acro.xml; Record Model and Filter Module</title>
- <warning>
- <para>
- The functionality of this record model has been improved and
- replaced by the DOM &acro.xml; record model, see
- <xref linkend="record-model-domxml"/>. The Alvis &acro.xml; record
- model is considered obsolete, and will eventually be removed
- from future releases of the &zebra; software.
- </para>
- </warning>
+ <warning>
+ <para>
+ The functionality of this record model has been improved and
+ replaced by the DOM &acro.xml; record model, see
+ <xref linkend="record-model-domxml"/>. The Alvis &acro.xml; record
+ model is considered obsolete, and will eventually be removed
+ from future releases of the &zebra; software.
+ </para>
+ </warning>
<para>
The record model described in this chapter applies to the fundamental,
<xref linkend="componentmodulesalvis"/>.
</para>
- <para> This filter has been developed under the
+ <para> This filter has been developed under the
<ulink url="http://www.alvis.info/">ALVIS</ulink> project funded by
the European Community under the "Information Society Technologies"
Program (2002-2006).
</para>
-
-
+
+
<section id="record-model-alvisxslt-filter">
<title>ALVIS Record Filter</title>
<para>
The experimental, loadable Alvis &acro.xml;/&acro.xslt; filter module
- <literal>mod-alvis.so</literal> is packaged in the GNU/Debian package
+ <literal>mod-alvis.so</literal> is packaged in the GNU/Debian package
<literal>libidzebra1.4-mod-alvis</literal>.
It is invoked by the <filename>zebra.cfg</filename> configuration statement
<screen>
recordtype.xml: alvis.db/filter_alvis_conf.xml
</screen>
- In this example on all data files with suffix
+ In this example on all data files with suffix
<filename>*.xml</filename>, where the
Alvis &acro.xslt; filter configuration file is found in the
path <filename>db/filter_alvis_conf.xml</filename>.
valid &acro.xml;. It might look like this (This example is
used for indexing and display of &acro.oai; harvested records):
<screen>
- <?xml version="1.0" encoding="UTF-8"?>
- <schemaInfo>
- <schema name="identity" stylesheet="xsl/identity.xsl" />
- <schema name="index" identifier="http://indexdata.dk/zebra/xslt/1"
- stylesheet="xsl/oai2index.xsl" />
- <schema name="dc" stylesheet="xsl/oai2dc.xsl" />
- <!-- use split level 2 when indexing whole OAI Record lists -->
- <split level="2"/>
- </schemaInfo>
- </screen>
+ <?xml version="1.0" encoding="UTF-8"?>
+ <schemaInfo>
+ <schema name="identity" stylesheet="xsl/identity.xsl" />
+ <schema name="index" identifier="http://indexdata.dk/zebra/xslt/1"
+ stylesheet="xsl/oai2index.xsl" />
+ <schema name="dc" stylesheet="xsl/oai2dc.xsl" />
+ <!-- use split level 2 when indexing whole OAI Record lists -->
+ <split level="2"/>
+ </schemaInfo>
+ </screen>
</para>
<para>
All named stylesheets defined inside
- <literal>schema</literal> element tags
+ <literal>schema</literal> element tags
are for presentation after search, including
the indexing stylesheet (which is a great debugging help). The
names defined in the <literal>name</literal> attributes must be
- unique, these are the literal <literal>schema</literal> or
- <literal>element set</literal> names used in
- <ulink url="http://www.loc.gov/standards/sru/srw/">&acro.srw;</ulink>,
- <ulink url="&url.sru;">&acro.sru;</ulink> and
+ unique, these are the literal <literal>schema</literal> or
+ <literal>element set</literal> names used in
+ <ulink url="http://www.loc.gov/standards/sru/srw/">&acro.srw;</ulink>,
+ <ulink url="&url.sru;">&acro.sru;</ulink> and
&acro.z3950; protocol queries.
The paths in the <literal>stylesheet</literal> attributes
are relative to zebras working directory, or absolute to file
</para>
<para>
There must be exactly one indexing &acro.xslt; stylesheet, which is
- defined by the magic attribute
+ defined by the magic attribute
<literal>identifier="http://indexdata.dk/zebra/xslt/1"</literal>.
</para>
<section id="record-model-alvisxslt-internal">
- <title>ALVIS Internal Record Representation</title>
+ <title>ALVIS Internal Record Representation</title>
<para>When indexing, an &acro.xml; Reader is invoked to split the input
- files into suitable record &acro.xml; pieces. Each record piece is then
- transformed to an &acro.xml; &acro.dom; structure, which is essentially the
- record model. Only &acro.xslt; transformations can be applied during
- index, search and retrieval. Consequently, output formats are
- restricted to whatever &acro.xslt; can deliver from the record &acro.xml;
- structure, be it other &acro.xml; formats, HTML, or plain text. In case
- you have <literal>libxslt1</literal> running with E&acro.xslt; support,
- you can use this functionality inside the Alvis
- filter configuration &acro.xslt; stylesheets.
+ files into suitable record &acro.xml; pieces. Each record piece is then
+ transformed to an &acro.xml; &acro.dom; structure, which is essentially the
+ record model. Only &acro.xslt; transformations can be applied during
+ index, search and retrieval. Consequently, output formats are
+ restricted to whatever &acro.xslt; can deliver from the record &acro.xml;
+ structure, be it other &acro.xml; formats, HTML, or plain text. In case
+ you have <literal>libxslt1</literal> running with E&acro.xslt; support,
+ you can use this functionality inside the Alvis
+ filter configuration &acro.xslt; stylesheets.
</para>
</section>
<section id="record-model-alvisxslt-canonical">
- <title>ALVIS Canonical Indexing Format</title>
+ <title>ALVIS Canonical Indexing Format</title>
<para>The output of the indexing &acro.xslt; stylesheets must contain
- certain elements in the magic
+ certain elements in the magic
<literal>xmlns:z="http://indexdata.dk/zebra/xslt/1"</literal>
- namespace. The output of the &acro.xslt; indexing transformation is then
- parsed using &acro.dom; methods, and the contained instructions are
- performed on the <emphasis>magic elements and their
- subtrees</emphasis>.
+ namespace. The output of the &acro.xslt; indexing transformation is then
+ parsed using &acro.dom; methods, and the contained instructions are
+ performed on the <emphasis>magic elements and their
+ subtrees</emphasis>.
</para>
<para>
- For example, the output of the command
- <screen>
+ For example, the output of the command
+ <screen>
xsltproc xsl/oai2index.xsl one-record.xml
- </screen>
+ </screen>
might look like this:
<screen>
<?xml version="1.0" encoding="UTF-8"?>
- <z:record xmlns:z="http://indexdata.dk/zebra/xslt/1"
- z:id="oai:JTRS:CP-3290---Volume-I"
- z:rank="47896">
- <z:index name="oai_identifier" type="0">
- oai:JTRS:CP-3290---Volume-I</z:index>
- <z:index name="oai_datestamp" type="0">2004-07-09</z:index>
- <z:index name="oai_setspec" type="0">jtrs</z:index>
- <z:index name="dc_all" type="w">
- <z:index name="dc_title" type="w">Proceedings of the 4th
- International Conference and Exhibition:
- World Congress on Superconductivity - Volume I</z:index>
- <z:index name="dc_creator" type="w">Kumar Krishen and *Calvin
- Burnham, Editors</z:index>
- </z:index>
- </z:record>
+ <z:record xmlns:z="http://indexdata.dk/zebra/xslt/1"
+ z:id="oai:JTRS:CP-3290---Volume-I"
+ z:rank="47896">
+ <z:index name="oai_identifier" type="0">
+ oai:JTRS:CP-3290---Volume-I</z:index>
+ <z:index name="oai_datestamp" type="0">2004-07-09</z:index>
+ <z:index name="oai_setspec" type="0">jtrs</z:index>
+ <z:index name="dc_all" type="w">
+ <z:index name="dc_title" type="w">Proceedings of the 4th
+ International Conference and Exhibition:
+ World Congress on Superconductivity - Volume I</z:index>
+ <z:index name="dc_creator" type="w">Kumar Krishen and *Calvin
+ Burnham, Editors</z:index>
+ </z:index>
+ </z:record>
</screen>
</para>
- <para>This means the following: From the original &acro.xml; file
+ <para>This means the following: From the original &acro.xml; file
<literal>one-record.xml</literal> (or from the &acro.xml; record &acro.dom; of the
same form coming from a split input file), the indexing
stylesheet produces an indexing &acro.xml; record, which is defined by
the <literal>record</literal> element in the magic namespace
<literal>xmlns:z="http://indexdata.dk/zebra/xslt/1"</literal>.
- &zebra; uses the content of
+ &zebra; uses the content of
<literal>z:id="oai:JTRS:CP-3290---Volume-I"</literal> as internal
- record ID, and - in case static ranking is set - the content of
+ record ID, and - in case static ranking is set - the content of
<literal>z:rank="47896"</literal> as static rank. Following the
discussion in <xref linkend="administration-ranking"/>
we see that this records is internally ordered
<literal>oai:JTRS:CP-3290---Volume-I47896</literal>.
<!-- The type of action performed during indexing is defined by
<literal>z:type="update"></literal>, with recognized values
- <literal>insert</literal>, <literal>update</literal>, and
- <literal>delete</literal>. -->
+ <literal>insert</literal>, <literal>update</literal>, and
+ <literal>delete</literal>. -->
</para>
<para>In this example, the following literal indexes are constructed:
<screen>
- oai_identifier
- oai_datestamp
- oai_setspec
- dc_all
- dc_title
- dc_creator
+ oai_identifier
+ oai_datestamp
+ oai_setspec
+ dc_all
+ dc_title
+ dc_creator
</screen>
- where the indexing type is defined in the
- <literal>type</literal> attribute
+ where the indexing type is defined in the
+ <literal>type</literal> attribute
(any value from the standard configuration
- file <filename>default.idx</filename> will do). Finally, any
+ file <filename>default.idx</filename> will do). Finally, any
<literal>text()</literal> node content recursively contained
inside the <literal>index</literal> will be filtered through the
appropriate char map for character normalization, and will be
<literal>oai:JTRS:CP-3290---Volume-I</literal> will be literal,
byte for byte without any form of character normalization,
inserted into the index named <literal>oai:identifier</literal>,
- the text
+ the text
<literal>Kumar Krishen and *Calvin Burnham, Editors</literal>
will be inserted using the <literal>w</literal> character
normalization defined in <filename>default.idx</filename> into
the index <literal>dc:creator</literal> (that is, after character
- normalization the index will keep the individual words
- <literal>kumar</literal>, <literal>krishen</literal>,
+ normalization the index will keep the individual words
+ <literal>kumar</literal>, <literal>krishen</literal>,
<literal>and</literal>, <literal>calvin</literal>,
<literal>burnham</literal>, and <literal>editors</literal>), and
finally both the texts
<literal>Proceedings of the 4th International Conference and Exhibition:
- World Congress on Superconductivity - Volume I</literal>
+ World Congress on Superconductivity - Volume I</literal>
and
- <literal>Kumar Krishen and *Calvin Burnham, Editors</literal>
+ <literal>Kumar Krishen and *Calvin Burnham, Editors</literal>
will be inserted into the index <literal>dc:all</literal> using
- the same character normalization map <literal>w</literal>.
+ the same character normalization map <literal>w</literal>.
</para>
<para>
Finally, this example configuration can be queried using &acro.pqf;
- queries, either transported by &acro.z3950;, (here using a yaz-client)
+ queries, either transported by &acro.z3950;, (here using a yaz-client)
<screen>
<![CDATA[
Z> open localhost:9999
<title>ALVIS Record Model Configuration</title>
- <section id="record-model-alvisxslt-index">
- <title>ALVIS Indexing Configuration</title>
+ <section id="record-model-alvisxslt-index">
+ <title>ALVIS Indexing Configuration</title>
<para>
As mentioned above, there can be only one indexing
stylesheet, and configuration of the indexing process is a synonym
of writing an &acro.xslt; stylesheet which produces &acro.xml; output containing the
- magic elements discussed in
- <xref linkend="record-model-alvisxslt-internal"/>.
+ magic elements discussed in
+ <xref linkend="record-model-alvisxslt-internal"/>.
Obviously, there are million of different ways to accomplish this
task, and some comments and code snippets are in order to lead
our Padawan's on the right track to the good side of the force.
<emphasis>push</emphasis> type might be the only possible way to
sort out deeply recursive input &acro.xml; formats.
</para>
- <para>
+ <para>
A <emphasis>pull</emphasis> stylesheet example used to index
&acro.oai; harvested records could use some of the following template
definitions:
<screen>
<![CDATA[
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- xmlns:z="http://indexdata.dk/zebra/xslt/1"
- xmlns:oai="http://www.openarchives.org/&acro.oai;/2.0/"
- xmlns:oai_dc="http://www.openarchives.org/&acro.oai;/2.0/oai_dc/"
- xmlns:dc="http://purl.org/dc/elements/1.1/"
- version="1.0">
-
- <xsl:output indent="yes" method="xml" version="1.0" encoding="UTF-8"/>
-
- <!-- disable all default text node output -->
- <xsl:template match="text()"/>
-
- <!-- match on oai xml record root -->
- <xsl:template match="/">
- <z:record z:id="{normalize-space(oai:record/oai:header/oai:identifier)}">
- <!-- you might want to use z:rank="{some &acro.xslt; function here}" -->
- <xsl:apply-templates/>
- </z:record>
- </xsl:template>
-
- <!-- &acro.oai; indexing templates -->
- <xsl:template match="oai:record/oai:header/oai:identifier">
- <z:index name="oai_identifier" type="0">
- <xsl:value-of select="."/>
- </z:index>
- </xsl:template>
-
- <!-- etc, etc -->
-
- <!-- DC specific indexing templates -->
- <xsl:template match="oai:record/oai:metadata/oai_dc:dc/dc:title">
- <z:index name="dc_title" type="w">
- <xsl:value-of select="."/>
- </z:index>
- </xsl:template>
-
- <!-- etc, etc -->
-
- </xsl:stylesheet>
+ xmlns:z="http://indexdata.dk/zebra/xslt/1"
+ xmlns:oai="http://www.openarchives.org/&acro.oai;/2.0/"
+ xmlns:oai_dc="http://www.openarchives.org/&acro.oai;/2.0/oai_dc/"
+ xmlns:dc="http://purl.org/dc/elements/1.1/"
+ version="1.0">
+
+ <xsl:output indent="yes" method="xml" version="1.0" encoding="UTF-8"/>
+
+ <!-- disable all default text node output -->
+ <xsl:template match="text()"/>
+
+ <!-- match on oai xml record root -->
+ <xsl:template match="/">
+ <z:record z:id="{normalize-space(oai:record/oai:header/oai:identifier)}">
+ <!-- you might want to use z:rank="{some &acro.xslt; function here}" -->
+ <xsl:apply-templates/>
+ </z:record>
+ </xsl:template>
+
+ <!-- &acro.oai; indexing templates -->
+ <xsl:template match="oai:record/oai:header/oai:identifier">
+ <z:index name="oai_identifier" type="0">
+ <xsl:value-of select="."/>
+ </z:index>
+ </xsl:template>
+
+ <!-- etc, etc -->
+
+ <!-- DC specific indexing templates -->
+ <xsl:template match="oai:record/oai:metadata/oai_dc:dc/dc:title">
+ <z:index name="dc_title" type="w">
+ <xsl:value-of select="."/>
+ </z:index>
+ </xsl:template>
+
+ <!-- etc, etc -->
+
+ </xsl:stylesheet>
]]>
</screen>
</para>
Notice also,
that the names and types of the indexes can be defined in the
indexing &acro.xslt; stylesheet <emphasis>dynamically according to
- content in the original &acro.xml; records</emphasis>, which has
+ content in the original &acro.xml; records</emphasis>, which has
opportunities for great power and wizardry as well as grande
- disaster.
+ disaster.
</para>
<para>
The following excerpt of a <emphasis>push</emphasis> stylesheet
- <emphasis>might</emphasis>
+ <emphasis>might</emphasis>
be a good idea according to your strict control of the &acro.xml;
input format (due to rigorous checking against well-defined and
tight RelaxNG or &acro.xml; Schema's, for example):
<screen>
<![CDATA[
- <xsl:template name="element-name-indexes">
- <z:index name="{name()}" type="w">
- <xsl:value-of select="'1'"/>
- </z:index>
- </xsl:template>
+ <xsl:template name="element-name-indexes">
+ <z:index name="{name()}" type="w">
+ <xsl:value-of select="'1'"/>
+ </z:index>
+ </xsl:template>
]]>
</screen>
- This template creates indexes which have the name of the working
+ This template creates indexes which have the name of the working
node of any input &acro.xml; file, and assigns a '1' to the index.
- The example query
- <literal>find @attr 1=xyz 1</literal>
+ The example query
+ <literal>find @attr 1=xyz 1</literal>
finds all files which contain at least one
<literal>xyz</literal> &acro.xml; element. In case you can not control
which element names the input files contain, you might ask for
</para>
<para>
One variation over the theme <emphasis>dynamically created
- indexes</emphasis> will definitely be unwise:
+ indexes</emphasis> will definitely be unwise:
<screen>
- <![CDATA[
+ <![CDATA[
<!-- match on oai xml record root -->
- <xsl:template match="/">
- <z:record>
-
- <!-- create dynamic index name from input content -->
- <xsl:variable name="dynamic_content">
- <xsl:value-of select="oai:record/oai:header/oai:identifier"/>
- </xsl:variable>
-
- <!-- create zillions of indexes with unknown names -->
- <z:index name="{$dynamic_content}" type="w">
- <xsl:value-of select="oai:record/oai:metadata/oai_dc:dc"/>
- </z:index>
- </z:record>
-
- </xsl:template>
+ <xsl:template match="/">
+ <z:record>
+
+ <!-- create dynamic index name from input content -->
+ <xsl:variable name="dynamic_content">
+ <xsl:value-of select="oai:record/oai:header/oai:identifier"/>
+ </xsl:variable>
+
+ <!-- create zillions of indexes with unknown names -->
+ <z:index name="{$dynamic_content}" type="w">
+ <xsl:value-of select="oai:record/oai:metadata/oai_dc:dc"/>
+ </z:index>
+ </z:record>
+
+ </xsl:template>
]]>
</screen>
Don't be tempted to cross
to suffering and pain, and universal
disintegration of your project schedule.
</para>
- </section>
+ </section>
- <section id="record-model-alvisxslt-elementset">
- <title>ALVIS Exchange Formats</title>
- <para>
+ <section id="record-model-alvisxslt-elementset">
+ <title>ALVIS Exchange Formats</title>
+ <para>
An exchange format can be anything which can be the outcome of an
&acro.xslt; transformation, as far as the stylesheet is registered in
the main Alvis &acro.xslt; filter configuration file, see
<xref linkend="record-model-alvisxslt-filter"/>.
In principle anything that can be expressed in &acro.xml;, HTML, and
- TEXT can be the output of a <literal>schema</literal> or
- <literal>element set</literal> directive during search, as long as
- the information comes from the
+ TEXT can be the output of a <literal>schema</literal> or
+ <literal>element set</literal> directive during search, as long as
+ the information comes from the
<emphasis>original input record &acro.xml; &acro.dom; tree</emphasis>
(and not the transformed and <emphasis>indexed</emphasis> &acro.xml;!!).
</para>
indexer can be accessed during record retrieval. The following
example is a summary of the possibilities:
<screen>
- <![CDATA[
+ <![CDATA[
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- xmlns:z="http://indexdata.dk/zebra/xslt/1"
- version="1.0">
-
- <!-- register internal zebra parameters -->
- <xsl:param name="id" select="''"/>
- <xsl:param name="filename" select="''"/>
- <xsl:param name="score" select="''"/>
- <xsl:param name="schema" select="''"/>
-
- <xsl:output indent="yes" method="xml" version="1.0" encoding="UTF-8"/>
-
- <!-- use then for display of internal information -->
- <xsl:template match="/">
- <z:zebra>
- <id><xsl:value-of select="$id"/></id>
- <filename><xsl:value-of select="$filename"/></filename>
- <score><xsl:value-of select="$score"/></score>
- <schema><xsl:value-of select="$schema"/></schema>
- </z:zebra>
- </xsl:template>
-
- </xsl:stylesheet>
+ xmlns:z="http://indexdata.dk/zebra/xslt/1"
+ version="1.0">
+
+ <!-- register internal zebra parameters -->
+ <xsl:param name="id" select="''"/>
+ <xsl:param name="filename" select="''"/>
+ <xsl:param name="score" select="''"/>
+ <xsl:param name="schema" select="''"/>
+
+ <xsl:output indent="yes" method="xml" version="1.0" encoding="UTF-8"/>
+
+ <!-- use then for display of internal information -->
+ <xsl:template match="/">
+ <z:zebra>
+ <id><xsl:value-of select="$id"/></id>
+ <filename><xsl:value-of select="$filename"/></filename>
+ <score><xsl:value-of select="$score"/></score>
+ <schema><xsl:value-of select="$schema"/></schema>
+ </z:zebra>
+ </xsl:template>
+
+ </xsl:stylesheet>
]]>
</screen>
</para>
- </section>
+ </section>
- <section id="record-model-alvisxslt-example">
- <title>ALVIS Filter &acro.oai; Indexing Example</title>
- <para>
+ <section id="record-model-alvisxslt-example">
+ <title>ALVIS Filter &acro.oai; Indexing Example</title>
+ <para>
The source code tarball contains a working Alvis filter example in
the directory <filename>examples/alvis-oai/</filename>, which
- should get you started.
+ should get you started.
</para>
<para>
More example data can be harvested from any &acro.oai; compliant server,
- see details at the &acro.oai;
+ see details at the &acro.oai;
<ulink url="http://www.openarchives.org/">
http://www.openarchives.org/</ulink> web site, and the community
- links at
+ links at
<ulink url="http://www.openarchives.org/community/index.html">
http://www.openarchives.org/community/index.html</ulink>.
There is a tutorial
</section>
-
+
</chapter>
sgml-always-quote-attributes:t
sgml-indent-step:1
sgml-indent-data:t
- sgml-parent-document: "zebra.xml"
+ sgml-parent-document: "idzebra.xml"
sgml-local-catalogs: nil
sgml-namecase-general:t
End:
projects / idzebra-moved-to-github.git / blobdiff