X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=doc%2Fbook.xml;h=2434c840ee9c8f660951e48d1792e56cb96d9332;hb=383c1010e6cdbffebdce97c98c9f4e205717e613;hp=566a9ee29f2189ab42700ea0a3ae29c8bddc405a;hpb=c1d953eee6c00432493bc364da6284704ccd9cc2;p=metaproxy-moved-to-github.git

diff --git a/doc/book.xml b/doc/book.xml
index 566a9ee..2434c84 100644
--- a/doc/book.xml
+++ b/doc/book.xml
@@ -1,6 +1,6 @@
 <?xml version="1.0" standalone="no"?>
 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
-    "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" 
+    "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
 [
      <!ENTITY copyright SYSTEM "copyright.xml">
      <!ENTITY % local SYSTEM "local.ent">
@@ -26,7 +26,7 @@
   </authorgroup>
   <releaseinfo>&version;</releaseinfo>
   <copyright>
-   <year>2005-2011</year>
+   <year>2005-2014</year>
    <holder>Index Data</holder>
   </copyright>
   <abstract>
@@ -39,10 +39,10 @@
     processes, interprets and redirects requests from IR clients using
     standard protocols such as the binary
     <ulink url="&url.z39.50;">ANSI/NISO Z39.50</ulink>
-    and  the information search and retrieval 
+    and  the information search and retrieval
     web service <ulink url="&url.sru;">SRU</ulink>
     as well as functioning as a limited
-    <ulink url="&url.http;">HTTP</ulink> server. 
+    <ulink url="&url.http;">HTTP</ulink> server.
    </simpara>
    <simpara>
     Metaproxy is configured by an XML file which
@@ -72,7 +72,7 @@
 
  <chapter id="introduction">
   <title>Introduction</title>
-  
+
   <para>
    <ulink url="&url.metaproxy;">Metaproxy</ulink>
    is a stand alone program that acts as a universal router, proxy and
@@ -80,7 +80,7 @@
    as <ulink url="&url.z39.50;">Z39.50</ulink> and
    <ulink url="&url.sru;">SRU</ulink>.
    To clients, it acts as a server of these protocols: it can be searched,
-   records can be retrieved from it, etc. 
+   records can be retrieved from it, etc.
    To servers, it acts as a client: it searches in them,
    retrieves records from them, etc.  it satisfies its clients'
    requests by transforming them, multiplexing them, forwarding them
@@ -123,7 +123,7 @@
    including examples.
   </para>
  </chapter>
- 
+
  <chapter id="installation">
   <title>Installation</title>
   <para>
@@ -138,7 +138,7 @@
     </varlistentry>
     <varlistentry><term><ulink url="&url.libxslt;">Libxslt</ulink></term>
      <listitem>
-      <para>This is an XSLT processor - based on 
+      <para>This is an XSLT processor - based on
        <ulink url="&url.libxml2;">Libxml2</ulink>. Both Libxml2 and
        Libxslt must be installed with the development components
        (header files, etc.) as well as the run-time libraries.
@@ -150,7 +150,7 @@
       <para>
        The popular C++ library. Initial versions of Metaproxy
        was built with 1.32 but this is no longer supported.
-       Metaproxy is known to work with Boost version 1.33 through 1.38.
+       Metaproxy is known to work with Boost version 1.33 through 1.55.
       </para>
      </listitem>
     </varlistentry>
@@ -166,9 +166,14 @@
   <para>
    We have successfully built Metaproxy using the compilers
    <ulink url="&url.gcc;">GCC</ulink> version 4.0 and
-   <ulink url="&url.vstudio;">Microsoft Visual Studio</ulink> 2003/2005.
+   <ulink url="&url.vstudio;">Microsoft Visual Studio</ulink> 2003/2005/2008.
   </para>
 
+  <para>
+   As an option, Metaproxy may also be compiled with
+   <ulink url="&url.usemarcon;">USEMARCON</ulink> support which allows for
+   MARC conversions for the <xref linkend="ref-record_transform"/> filter.
+  </para>
   <section id="installation.unix">
    <title>Installation on Unix (from Source)</title>
    <para>
@@ -178,6 +183,14 @@
     installed as development packages use those (and omit compilation).
    </para>
 
+   <note>
+    <para>
+     <ulink url="&url.usemarcon;">USEMARCON</ulink> is not available
+     as a package at the moment, so Metaproxy must be built from source
+     if that is to be used.
+    </para>
+   </note>
+
    <section id="libxml2.fromsource">
     <title>Libxml2/libxslt</title>
     <para>
@@ -200,6 +213,18 @@
      make install
     </screen>
    </section>
+   <section id="usemarcon">
+    <title>USEMARCON (optional)</title>
+    <screen>
+     gunzip -c usemarcon317.tar.gz|tar xf -
+     cd usemarcon317
+     ./configure
+     make
+     su
+     make install
+    </screen>
+   </section>
+
    <section id="yaz.fromsource">
     <title>YAZ/YAZ++</title>
     <screen>
@@ -228,7 +253,7 @@
     <screen>
      gunzip -c boost-version.tar.gz|tar xf -
      cd boost-version
-     ./configure --with-libraries=thread,test --with-toolset=gcc
+     ./configure --with-libraries=thread,test,regex --with-toolset=gcc
      make
      su
      make install
@@ -237,7 +262,7 @@
      However, under the hood bjam is used. You can invoke that with
     </para>
     <screen>
-     ./bjam --toolset=gcc --with-thread --with-test stage
+     ./bjam --toolset=gcc --with-thread --with-test --with-regex stage
     </screen>
     <para>
      Replace <literal>stage</literal> with <literal>clean</literal> /
@@ -265,7 +290,7 @@
      The latter the compiler toolset (eg. gcc34).
     </para>
     <para>
-     Pass <literal>--help</literal> to configure to get a list of 
+     Pass <literal>--help</literal> to configure to get a list of
      available options.
     </para>
    </section>
@@ -274,10 +299,8 @@
   <section id="installation.debian">
    <title>Installation on Debian GNU/Linux</title>
    <para>
-    All dependencies for Metaproxy are available as 
-    <ulink url="&url.debian;">Debian</ulink>
-    packages for the sarge (stable in 2005) and etch (testing in 2005)
-    distributions.
+    All dependencies for Metaproxy are available as
+    <ulink url="&url.debian;">Debian</ulink> packages.
    </para>
    <para>
     The procedures for Debian based systems, such as
@@ -286,7 +309,7 @@
    <para>
     There is currently no official Debian package for YAZ++.
     And the official Debian package for YAZ is probably too old.
-    But Index Data bulds "new" versions of those for Debian (i386 only).
+    But Index Data builds "new" versions of those for Debian (i386, amd64 only).
    </para>
    <para>
     Update the <filename>/etc/apt/sources.list</filename>
@@ -296,10 +319,12 @@
    </para>
    <screen>
     apt-get install libxslt1-dev
-    apt-get install libyazpp2-dev
+    apt-get install libyazpp6-dev
     apt-get install libboost-dev
+    apt-get install libboost-system-dev
     apt-get install libboost-thread-dev
     apt-get install libboost-test-dev
+    apt-get install libboost-regex-dev
    </screen>
    <para>
     With these packages installed, the usual configure + make
@@ -311,14 +336,14 @@
   <section id="installation.rpm">
    <title>Installation on RPM based Linux Systems</title>
    <para>
-    All external dependencies for Metaproxy are available as 
-    RPM packages, either from your distribution site, or from the 
+    All external dependencies for Metaproxy are available as
+    RPM packages, either from your distribution site, or from the
     <ulink url="http://fr.rpmfind.net/">RPMfind</ulink> site.
    </para>
    <para>
-    For example, an installation of the requires Boost C++ development 
+    For example, an installation of the requires Boost C++ development
     libraries on RedHat Fedora C4 and C5 can be done like this:
-    <screen> 
+    <screen>
     wget ftp://fr.rpmfind.net/wlinux/fedora/core/updates/testing/4/SRPMS/boost-1.33.0-3.fc4.src.rpm
     sudo rpmbuild --buildroot src/ --rebuild -p fc4/boost-1.33.0-3.fc4.src.rpm
     sudo rpm -U /usr/src/redhat/RPMS/i386/boost-*rpm
@@ -331,7 +356,7 @@
    </para>
    <para>
     There is currently no official RPM package for YAZ++.
-    See the <ulink url="&url.yazplusplus;">YAZ++</ulink> pages 
+    See the <ulink url="&url.yazplusplus;">YAZ++</ulink> pages
     for more information on a Unix tarball install.
    </para>
    <para>
@@ -346,7 +371,7 @@
    <para>
     Metaproxy can be compiled with Microsoft
     <ulink url="&url.vstudio;">Visual Studio</ulink>.
-    Versions 2003 (C 7.1), 2005 (C 8.0) and 2008 (C 9.0) is known to work.
+    Versions 2003 (C 7.1), 2005 (C 8.0) and 2008 (C 9.0) are known to work.
    </para>
    <section id="installation.windows.boost">
     <title>Boost</title>
@@ -392,7 +417,7 @@
     <title>YAZ++</title>
     <para>
      Get <ulink url="&url.yazplusplus;">YAZ++</ulink> as well.
-     Version 1.1.0 or later is required.
+     Version 1.5.2 or later is required.
     </para>
     <para>
      YAZ++ includes NMAKE makefiles, similar to those found in the
@@ -457,9 +482,9 @@
        </para>
       </listitem>
      </varlistentry>
-      
+
     </variablelist>
-    
+
     <para>
      After successful compilation you'll find
      <literal>metaproxy.exe</literal> in the
@@ -470,7 +495,7 @@
 
   </section>
  </chapter>
- 
+
 <chapter id="yazproxy-comparison">
  <title>YAZ Proxy Comparison</title>
  <para>
@@ -535,7 +560,7 @@
    </row>
    <row>
     <entry>SRU Virtual database, i.e. select any Z39.50 target for path</entry>
-    <entry>Supported with filter <literal>virt_db</literal>, 
+    <entry>Supported with filter <literal>virt_db</literal>,
      <literal>sru_z3950</literal></entry>
     <entry>Supported</entry>
    </row>
@@ -597,7 +622,7 @@
 
    <row>
     <entry><ulink url="&url.usemarcon;">USEMARCON</ulink></entry>
-    <entry>Unsupported</entry>
+    <entry>Supported with <literal>record_transform</literal></entry>
     <entry>Supported</entry>
    </row>
 
@@ -716,8 +741,8 @@
 
  <chapter id="filters">
   <title>Filters</title>
-  
-  
+
+
   <section id="filters-introductory-notes">
    <title>Introductory notes</title>
    <para>
@@ -755,7 +780,7 @@
     others are sinks: they consume packages and return a result
     (<literal>backend_test</literal>,
     <literal>bounce</literal>,
-    <literal>http_file</literal>, 
+    <literal>http_file</literal>,
     <literal>z3950_client</literal>);
     the others are true filters, that read, process and pass on the
     packages they are fed
@@ -770,8 +795,8 @@
     <literal>virt_db</literal>).
    </para>
  </section>
-  
-  
+
+
   <section id="overview.filter.types">
    <title>Overview of filter types</title>
    <para>
@@ -793,7 +818,7 @@
    <para>
     The filters are here listed in alphabetical order:
    </para>
-   
+
 <!--
 
 ### New filters:
@@ -835,7 +860,7 @@ Figure out what additional information we need in:
      the user.
     </para>
    </section>
-   
+
    <section id="backend_test">
     <title><literal>backend_test</literal>
     (mp::filter::Backend_test)</title>
@@ -846,38 +871,38 @@ Figure out what additional information we need in:
      even read this section.
     </para>
    </section>
-   
+
    <section id="bounce">
     <title><literal>bounce</literal>
     (mp::filter::Bounce)</title>
     <para>
-     A sink that swallows <emphasis>all packages</emphasis>, 
+     A sink that swallows <emphasis>all packages</emphasis>,
      and returns them almost unprocessed.
      It never sends any package of any type further down the row, but
      sets Z39.50 packages to Z_Close, and HTTP_Request packages to
      HTTP_Response err code 400 packages, and adds a suitable bounce
-     message. 
+     message.
      The bounce filter is usually added at end of each filter chain route
      to prevent infinite hanging of for example HTTP
-     requests packages when only the Z39.50 client partial sink 
+     requests packages when only the Z39.50 client partial sink
      filter is found in the
-     route.  
+     route.
     </para>
    </section>
-   
+
    <section id="cql_rpn">
     <title><literal>cql_rpn</literal>
     (mp::filter::CQLtoRPN)</title>
     <para>
-     A query language transforming filter which catches Z39.50 
+     A query language transforming filter which catches Z39.50
      <literal>searchRequest</literal>
      packages containing <literal>CQL</literal> queries, transforms
      those to <literal>RPN</literal> queries,
      and sends the <literal>searchRequests</literal> on to the next
-     filters. It is among other things useful in a SRU context. 
+     filters. It is among other things useful in a SRU context.
     </para>
    </section>
-   
+
    <section id="frontend_net">
     <title><literal>frontend_net</literal>
      (mp::filter::FrontendNet)</title>
@@ -893,19 +918,19 @@ Figure out what additional information we need in:
     <title><literal>http_file</literal>
      (mp::filter::HttpFile)</title>
     <para>
-     A partial sink which swallows only 
-     <literal>HTTP_Request</literal> packages, and 
+     A partial sink which swallows only
+     <literal>HTTP_Request</literal> packages, and
      returns the contents of files from the local
-     filesystem in response to HTTP requests.  
+     filesystem in response to HTTP requests.
      It lets Z39.50 packages and all other forthcoming package types
-     pass untouched. 
+     pass untouched.
      (Yes, Virginia, this
      does mean that Metaproxy is also a Web-server in its spare time.  So
      far it does not contain either an email-reader or a Lisp
      interpreter, but that day is surely coming.)
     </para>
    </section>
-   
+
    <section id="load_balance">
     <title><literal>load_balance</literal>
      (mp::filter::LoadBalance)</title>
@@ -913,7 +938,7 @@ Figure out what additional information we need in:
      Performs load balancing for incoming Z39.50 init requests.
      It is used together with the <literal>virt_db</literal> filter,
      but unlike the <literal>multi</literal> filter it does send an
-     entire session to only one of the virtual backends. The 
+     entire session to only one of the virtual backends. The
      <literal>load_balance</literal> filter is assuming that
      all backend targets have equal content, and chooses the backend
      with least load cost for a new session.
@@ -925,7 +950,7 @@ Figure out what additional information we need in:
     </warning>
    </para>
    </section>
-      
+
    <section id="log">
     <title><literal>log</literal>
      (mp::filter::Log)</title>
@@ -946,13 +971,13 @@ Figure out what additional information we need in:
      of virtual databases and multi-database searching below.
     </para>
    </section>
-   
+
    <section id="query_rewrite">
    <title><literal>query_rewrite</literal>
      (mp::filter::QueryRewrite)</title>
     <para>
-     Rewrites Z39.50 <literal>Type-1</literal> 
-     and <literal>Type-101</literal> (``<literal>RPN</literal>'') 
+     Rewrites Z39.50 <literal>Type-1</literal>
+     and <literal>Type-101</literal> (``<literal>RPN</literal>'')
      queries by a
      three-step process: the query is transliterated from Z39.50
      packet structures into an XML representation; that XML
@@ -961,8 +986,8 @@ Figure out what additional information we need in:
      structure.
     </para>
    </section>
-   
-   
+
+
    <section id="record_transform">
     <title><literal>record_transform</literal>
     (mp::filter::RecordTransform)</title>
@@ -984,7 +1009,7 @@ Figure out what additional information we need in:
      This filter implements global sharing of
      result sets (i.e. between threads and therefore between
      clients), yielding performance improvements by clever resource
-     pooling. 
+     pooling.
     </para>
    </section>
 
@@ -999,17 +1024,17 @@ Figure out what additional information we need in:
      messages.
      The <literal>sru_z3950</literal> filter  processes also  SRU
      GET/POST/SOAP explain requests, returning
-     either the absolute minimum required by the standard, or a  full 
+     either the absolute minimum required by the standard, or a  full
      pre-defined ZeeReX explain record.
-     See the 
+     See the
      <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
-     standard pages and the 
+     standard pages and the
      <ulink url="&url.sru.explain;">SRU Explain</ulink> pages
      for more information on the correct explain syntax.
      SRU scan requests are not supported yet.
     </para>
    </section>
-   
+
    <section id="template">
     <title><literal>template</literal>
      (mp::filter::Template)</title>
@@ -1022,7 +1047,7 @@ Figure out what additional information we need in:
      intended for civilians.
     </para>
    </section>
-   
+
    <section id="virt_db">
     <title><literal>virt_db</literal>
      (mp::filter::VirtualDB)</title>
@@ -1037,7 +1062,7 @@ Figure out what additional information we need in:
      of virtual databases and multi-database searching below.
     </para>
    </section>
-   
+
    <section id="z3950_client">
     <title><literal>z3950_client</literal>
      (mp::filter::Z3950Client)</title>
@@ -1051,7 +1076,7 @@ Figure out what additional information we need in:
      the route.  Subsequent requests are sent to the same address,
      which is remembered at Init time in a Session object.
      HTTP_Request packages and all other forthcoming package types
-     are passed untouched. 
+     are passed untouched.
     </para>
   </section>
 
@@ -1064,7 +1089,7 @@ Figure out what additional information we need in:
      Z39.50 explain requests, returning a static ZeeReX
      Explain XML record from the config section. All other packages
      are passed through.
-     See the 
+     See the
      <ulink url="&url.zeerex.explain;">ZeeReX Explain</ulink>
      standard pages
      for more information on the correct explain syntax.
@@ -1075,11 +1100,11 @@ Figure out what additional information we need in:
      </para>
     </warning>
    </section>
-   
+
 
   </section>
-  
-  
+
+
   <section id="future.directions">
    <title>Future directions</title>
   <para>
@@ -1117,13 +1142,13 @@ Figure out what additional information we need in:
    </variablelist>
   </section>
  </chapter>
- 
- 
- 
+
+
+
  <chapter id="configuration">
   <title>Configuration: the Metaproxy configuration file format</title>
-  
-  
+
+
   <section id="configuration-introductory-notes">
    <title>Introductory notes</title>
    <para>
@@ -1136,7 +1161,7 @@ Figure out what additional information we need in:
     Metaproxy.)
    </para>
   </section>
-  
+
   <section id="overview.xml.structure">
    <title>Overview of the config file XML structure</title>
    <para>
@@ -1158,7 +1183,7 @@ Figure out what additional information we need in:
     All four are non-repeatable.
    </para>
    <para>
-     The &lt;dlpath;&gt; element contains a text element which 
+     The &lt;dlpath;&gt; element contains a text element which
      specifies the location of filter modules. This is only needed
      if Metaproxy must load 3rd party filters (most filters with Metaproxy
      are built into the Metaproxy application).
@@ -1223,7 +1248,7 @@ Figure out what additional information we need in:
     <filter id="backend" type="z3950_client">
     </filter>
   </filters>
-  <routes>  
+  <routes>
     <route id="start">
       <filter refid="frontend"/>
       <filter type="log"/>
@@ -1258,7 +1283,7 @@ Figure out what additional information we need in:
     mutton, beef and trout packages.
     When the response arrives, it is handed
     back to the <literal>log</literal> filter, which emits another
-    message; and then to the <literal>frontend_net</literal> filter, 
+    message; and then to the <literal>frontend_net</literal> filter,
     which returns the response to the client.
    </para>
   </section>
@@ -1268,7 +1293,7 @@ Figure out what additional information we need in:
    <para>
     Metaproxy XML configuration snippets can be reused by other
     filters using the <literal>XInclude</literal> standard, as seen in
-    the <literal>/etc/config-sru-to-z3950.xml</literal> example SRU 
+    the <literal>/etc/config-sru-to-z3950.xml</literal> example SRU
     configuration.
    <screen><![CDATA[
     <filter id="sru" type="sru_z3950">
@@ -1286,7 +1311,7 @@ Figure out what additional information we need in:
    <para>
     The distribution contains RelaxNG Compact and XML syntax checking
     files, as well as XML Schema files. These are found in the
-    distribution paths 
+    distribution paths
    <screen>
     xml/schema/metaproxy.rnc
     xml/schema/metaproxy.rng
@@ -1296,8 +1321,8 @@ Figure out what additional information we need in:
     configuration files. For example, using the utility
     <filename>xmllint</filename>, syntax checking is done like this:
    <screen>
-    xmllint --noout --schema xml/schema/metaproxy.xsd etc/config-local.xml 
-    xmllint --noout --relaxng xml/schema/metaproxy.rng etc/config-local.xml 
+    xmllint --noout --schema xml/schema/metaproxy.xsd etc/config-local.xml
+    xmllint --noout --relaxng xml/schema/metaproxy.rng etc/config-local.xml
    </screen>
     (A recent version of <literal>libxml2</literal> is required, as
     support for XML Schemas is a relatively recent addition.)
@@ -1678,17 +1703,17 @@ Z>
  <chapter id="sru-server">
   <title>Combined SRU webservice and Z39.50 server configuration</title>
   <para>
-   Metaproxy can act as 
-   <ulink url="&url.sru;">SRU</ulink> and 
-   web service server, which translates web service requests to 
+   Metaproxy can act as
+   <ulink url="&url.sru;">SRU</ulink> and
+   web service server, which translates web service requests to
    <ulink url="&url.z39.50;">ANSI/NISO Z39.50</ulink> packages and
    sends them off to common available targets.
   </para>
   <para>
   A typical setup for this operation needs a filter route including the
-  following modules: 
+  following modules:
   </para>
-   
+
   <table id="sru-server-table-config" frame="top">
    <title>SRU/Z39.50 Server Filter Route Configuration</title>
    <tgroup cols="3">
@@ -1699,7 +1724,7 @@ Z>
       <entry>Purpose</entry>
      </row>
     </thead>
-    
+
     <tbody>
      <row>
       <entry><literal>frontend_net</literal></entry>
@@ -1769,16 +1794,16 @@ Z>
     </tbody>
    </tgroup>
   </table>
-  <para> 
+  <para>
    A typical minimal example <ulink url="&url.sru;">SRU</ulink>
-   server configuration file is found in the tarball distribution at 
+   server configuration file is found in the tarball distribution at
    <literal>etc/config-sru-to-z3950.xml</literal>.
-  </para> 
+  </para>
   <para>
    Off course, any other metaproxy modules can be integrated into a
-   SRU server solution, including, but not limited to, load balancing, 
-   multiple target querying 
-   (see  <xref linkend="multidb"/>), and complex RPN query rewrites. 
+   SRU server solution, including, but not limited to, load balancing,
+   multiple target querying
+   (see  <xref linkend="multidb"/>), and complex RPN query rewrites.
   </para>
 
 
@@ -2049,8 +2074,8 @@ Z>
    </para>
   </section>
  </chapter>
- 
- 
+
+
  <reference id="reference">
   <title>Reference</title>
    <partintro id="reference-introduction">
@@ -2083,30 +2108,22 @@ Z>
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.
   </para>
-  
+
   <para>
    You should have received a copy of the GNU General Public License
    along with Metaproxy; see the file LICENSE.  If not, write to the
-   Free Software Foundation, 
+   Free Software Foundation,
    51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
    </para>
-  
+
  </appendix>
 
  &gpl2;
 </book>
 
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: nil
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+<!-- Keep this comment at the end of the file
+Local variables:
+mode: nxml
+nxml-child-indent: 1
+End:
+-->