X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=doc%2Fbook.xml;h=d8f8e3c9c086caa08ac6e9f7b7d07eb43e34ca51;hb=b46d2379f2e3b7e0de3a94b8be2b3da20803a19a;hp=f65d15abc1bac50726c62e1b9e1a3f5580f1471e;hpb=620a7e3e1e566ac207e79bf080aa168a571d2be8;p=metaproxy-moved-to-github.git diff --git a/doc/book.xml b/doc/book.xml index f65d15a..d8f8e3c 100644 --- a/doc/book.xml +++ b/doc/book.xml @@ -1,25 +1,16 @@ - %local; - - - %common; - - - - + + + %idcommon; ]> - - + Metaproxy - User's Guide and Reference @@ -35,8 +26,8 @@ &version; - 2005-2007 - Index Data ApS + 2005-2012 + Index Data @@ -48,10 +39,9 @@ processes, interprets and redirects requests from IR clients using standard protocols such as the binary ANSI/NISO Z39.50 - and the information search and retireval - web services SRU - and SRW, as - well as functioning as a limited + and the information search and retrieval + web service SRU + as well as functioning as a limited HTTP server. @@ -65,10 +55,7 @@ using the filter API. - Metaproxy is not open-source software, but - may be freely downloaded, unpacked, inspected, built and run for - evaluation purposes. Deployment requires a separate, commercial, - license. + Metaproxy is covered by the GNU General Public License version 2. @@ -86,13 +73,12 @@ Introduction - Metaproxy is a stand alone program that acts as a universal router, proxy and encapsulated metasearcher for information retrieval protocols such - as Z39.50, and in the future - SRU and SRW. + as Z39.50 and + SRU. To clients, it acts as a server of these protocols: it can be searched, records can be retrieved from it, etc. To servers, it acts as a client: it searches in them, @@ -137,60 +123,6 @@ including examples. - - - The Metaproxy License - - - - You are allowed to download this software for evaluation purposes. - You can unpack it, build it, run it, see how it works and how it fits - your needs, all at zero cost. - - - - - You may NOT deploy the software. For the purposes of this license, - deployment means running it for any purpose other than evaluation, - whether or not you or anyone else makes a profit from doing so. If - you wish to deploy the software, you must first contact Index Data and - arrange to purchase a DEPLOYMENT LICENCE. If you are unsure - whether or not your proposed use of the software constitutes - deployment, email us at info@indexdata.com - for clarification. - - - - - You may modify your copy of the software (fix bugs, add features) - if you need to. We encourage you to send your changes back to us for - integration into the master copy, but you are not obliged to do so. You - may NOT pass your changes on to any other party. - - - - - There is NO WARRANTY for this software, to the extent permitted by - applicable law. We provide the software ``as is'' without warranty of - any kind, either expressed or implied, including, but not limited to, the - implied warranties of MERCHANTABILITY and FITNESS FOR A - PARTICULAR PURPOSE. The entire risk as to the quality and - performance of the software is with you. Should the software prove - defective, you assume the cost of all necessary servicing, repair or - correction. In no event unless required by applicable law will we be - liable to you for damages, arising out of the use of the software, - including but not limited to loss of data or data being rendered - inaccurate. - - - - - All rights to the software are reserved by Index Data except where - this license explicitly says otherwise. - - - - Installation @@ -217,7 +149,8 @@ The popular C++ library. Initial versions of Metaproxy - was built with 1.33.0. Version 1.33.1 works too. + was built with 1.32 but this is no longer supported. + Metaproxy is known to work with Boost version 1.33 through 1.46. @@ -233,7 +166,7 @@ We have successfully built Metaproxy using the compilers GCC version 4.0 and - Microsoft Visual Studio 2003/2005. + Microsoft Visual Studio 2003/2005/2008.
@@ -244,67 +177,98 @@ tools binary packages. If, for example, Libxml2/libxslt are already installed as development packages use those (and omit compilation). - - - Libxml2/libxslt: - - - gunzip -c libxml2-version.tar.gz|tar xf - - cd libxml2-version - ./configure - make - su - make install - - - gunzip -c libxslt-version.tar.gz|tar xf - - cd libxslt-version - ./configure - make - su - make install - - - YAZ/YAZ++: - - - gunzip -c yaz-version.tar.gz|tar xf - - cd yaz-version - ./configure - make - su - make install - - - gunzip -c yazpp-version.tar.gz|tar xf - - cd yazpp-version - ./configure - make - su - make install - - - Boost: - - - gunzip -c boost-version.tar.gz|tar xf - - cd boost-version - ./configure - make - su - make install - - - Metaproxy: - - - gunzip -c metaproxy-version.tar.gz|tar xf - - cd metaproxy-version - ./configure - make - su - make install - + +
+ Libxml2/libxslt + + Libxml2/libxslt: + + + gunzip -c libxml2-version.tar.gz|tar xf - + cd libxml2-version + ./configure + make + su + make install + + + gunzip -c libxslt-version.tar.gz|tar xf - + cd libxslt-version + ./configure + make + su + make install + +
+
+ YAZ/YAZ++ + + gunzip -c yaz-version.tar.gz|tar xf - + cd yaz-version + ./configure + make + su + make install + + + gunzip -c yazpp-version.tar.gz|tar xf - + cd yazpp-version + ./configure + make + su + make install + +
+
+ Boost + + Metaproxy needs components thread and test from + Boost. + + + gunzip -c boost-version.tar.gz|tar xf - + cd boost-version + ./configure --with-libraries=thread,test,regex --with-toolset=gcc + make + su + make install + + + However, under the hood bjam is used. You can invoke that with + + + ./bjam --toolset=gcc --with-thread --with-test --with-regex stage + + + Replace stage with clean / + install to perform clean and install respectively. + + + Add --prefix=DIR to install Boost in other + prefix than /usr/local. + +
+
+ Metaproxy + + gunzip -c metaproxy-version.tar.gz|tar xf - + cd metaproxy-version + ./configure + make + su + make install + + + You may have to tell configure where Boost is installed by supplying + options --with-boost and --with-boost-toolset. + The former sets the PREFIX for Boost (same as --prefix for Boost above). + The latter the compiler toolset (eg. gcc34). + + + Pass --help to configure to get a list of + available options. + +
@@ -321,7 +285,10 @@ There is currently no official Debian package for YAZ++. - And the Debian package for YAZ is probably too old. + And the official Debian package for YAZ is probably too old. + But Index Data builds "new" versions of those for Debian (i386, amd64 only). + + Update the /etc/apt/sources.list to include the Index Data repository. See YAZ' Download Debian @@ -329,12 +296,11 @@ apt-get install libxslt1-dev - apt-get install libyazpp-dev + apt-get install libyazpp4-dev apt-get install libboost-dev apt-get install libboost-thread-dev - apt-get install libboost-date-time-dev - apt-get install libboost-program-options-dev apt-get install libboost-test-dev + apt-get install libboost-regex-dev With these packages installed, the usual configure + make @@ -343,49 +309,55 @@
+
+ Installation on RPM based Linux Systems + + All external dependencies for Metaproxy are available as + RPM packages, either from your distribution site, or from the + RPMfind site. + + + For example, an installation of the requires Boost C++ development + libraries on RedHat Fedora C4 and C5 can be done like this: + + wget ftp://fr.rpmfind.net/wlinux/fedora/core/updates/testing/4/SRPMS/boost-1.33.0-3.fc4.src.rpm + sudo rpmbuild --buildroot src/ --rebuild -p fc4/boost-1.33.0-3.fc4.src.rpm + sudo rpm -U /usr/src/redhat/RPMS/i386/boost-*rpm + + + + The YAZ library is needed to + compile &metaproxy;, see there + for more information on available RPM packages. + + + There is currently no official RPM package for YAZ++. + See the YAZ++ pages + for more information on a Unix tarball install. + + + With these packages installed, the usual configure + make + procedure can be used for Metaproxy as outlined in + . + +
+
Installation on Windows Metaproxy can be compiled with Microsoft Visual Studio. - Version 2003 (C 7.1) and 2005 (C 8.0) is known to work. + Versions 2003 (C 7.1), 2005 (C 8.0) and 2008 (C 9.0) are known to work.
Boost - Get Boost from its home page. - You also need Boost Jam (an alternative to make). - That's also available from the Boost home page. - The files to be downloaded are called something like: - boost_1_33-1.exe - and - boost-jam-3.1.12-1-ntx86.zip. - Unpack Boost Jam first. Put bjam.exe - in your system path. Make a command prompt and ensure - it can be found automatically. If not check the PATH. - The Boost .exe is a self-extracting exe with - complete source for Boost. Compile that source with - Boost Jam (An alternative to Make). - The compilation takes a while. - For Visual Studio 2003, use - - bjam "-sTOOLS=vc-7_1" - - Here vc-7_1 refers to a "Toolset" (compiler system). - For Visual Studio 2005, use - - bjam "-sTOOLS=vc-8_0" - - To install the libraries in a common place, use - - bjam "-sTOOLS=vc-7_1" install - - (or vc-8_0 for VS 2005). - - - By default, the Boost build process installs the resulting - libraries + header files in - \boost\lib, \boost\include. + For Windows, it's easiest to get the precompiled Boost + package from here. + Several versions of the Boost libraries may be selected when + installing Boost for windows. Please choose at least the + multithreaded (non-DLL) version because + the Metaproxy makefile uses that. For more information about installing Boost refer to the @@ -403,7 +375,7 @@ Libxslt has other dependencies, but these can all be downloaded - from the same site. Get the following: + from the same site. Get the following package: iconv, zlib, libxml2, libxslt.
@@ -421,9 +393,7 @@ YAZ++ Get YAZ++ as well. - Version 1.0 or later is required. For now get it from - Index Data's - Snapshot area. + Version 1.2.7 or later is required. YAZ++ includes NMAKE makefiles, similar to those found in the @@ -502,6 +472,153 @@
+ + YAZ Proxy Comparison + + The table below lists facilities either supported by either + YAZ Proxy or Metaproxy. + + + Metaproxy / YAZ Proxy comparison + + + + Facility + Metaproxy + YAZ Proxy + + + + + Z39.50 server + Using filter frontend_net + Supported + + + SRU server + Supported with filter sru_z3950 + Supported + + + Z39.50 client + Supported with filter z3950_client + Supported + + + SRU client + Unsupported + Unsupported + + + Connection reuse + Supported with filter session_shared + Supported + + + Connection share + Supported with filter session_shared + Unsupported + + + Result set reuse + Supported with filter session_shared + Within one Z39.50 session / HTTP keep-alive + + + Record cache + Supported by filter session_shared + Supported for last result set within one Z39.50/HTTP-keep alive session + + + Z39.50 Virtual database, i.e. select any Z39.50 target for database + Supported with filter virt_db + Unsupported + + + SRU Virtual database, i.e. select any Z39.50 target for path + Supported with filter virt_db, + sru_z3950 + Supported + + + Multi target search + Supported with filter multi (round-robin) + Unsupported + + + Retrieval and search limits + Supported using filter limit + Supported + + + Bandwidth limits + Supported using filter limit + Supported + + + Connect limits + Supported by filter frontend_net (connect-max) + Supported + + + Retrieval sanity check and conversions + Supported using filter record_transform + Supported + + + Query check + + Supported by query_rewrite which may be check + a query and throw diagnostics (errors) + + Supported + + + Query rewrite + Supported with query_rewrite + Unsupported + + + Session invalidate for -1 hits + Unsupported + Supported + + + Architecture + Multi-threaded + select for networked modules such as + frontend_net) + Single-threaded using select + + + + Extensability + Most functionality implemented as loadable modules + Unsupported and experimental + + + + USEMARCON + Unsupported + Supported + + + + Portability + + Requires YAZ, YAZ++ and modern C++ compiler supporting + Boost. + + + Requires YAZ and YAZ++. + STL is not required so pretty much any C++ compiler out there should work. + + + + + +
+ + The Metaproxy Architecture @@ -582,7 +699,7 @@ plugins that provide new filters. The filter API is small and conceptually simple, but there are many details to master. See the section below on - extensions. + Filters. @@ -602,7 +719,7 @@ Filters -
+
Introductory notes It's useful to think of Metaproxy as an interpreter providing a small @@ -701,7 +818,7 @@ Figure out what additional information we need in: --> -
+
<literal>auth_simple</literal> (mp::filter::AuthSimple) @@ -720,7 +837,7 @@ Figure out what additional information we need in:
-
+
<literal>backend_test</literal> (mp::filter::Backend_test) @@ -731,7 +848,7 @@ Figure out what additional information we need in:
-
+
<literal>bounce</literal> (mp::filter::Bounce) @@ -749,7 +866,7 @@ Figure out what additional information we need in:
-
+
<literal>cql_rpn</literal> (mp::filter::CQLtoRPN) @@ -762,7 +879,7 @@ Figure out what additional information we need in:
-
+
<literal>frontend_net</literal> (mp::filter::FrontendNet) @@ -773,7 +890,7 @@ Figure out what additional information we need in:
-
+
<literal>http_file</literal> (mp::filter::HttpFile) @@ -790,7 +907,7 @@ Figure out what additional information we need in:
-
+
<literal>load_balance</literal> (mp::filter::LoadBalance) @@ -810,7 +927,7 @@ Figure out what additional information we need in:
-
+
<literal>log</literal> (mp::filter::Log) @@ -820,7 +937,7 @@ Figure out what additional information we need in:
-
+
<literal>multi</literal> (mp::filter::Multi) @@ -831,7 +948,7 @@ Figure out what additional information we need in:
-
+
<literal>query_rewrite</literal> (mp::filter::QueryRewrite) @@ -847,7 +964,7 @@ Figure out what additional information we need in:
-
+
<literal>record_transform</literal> (mp::filter::RecordTransform) @@ -861,7 +978,7 @@ Figure out what additional information we need in:
-
+
<literal>session_shared</literal> (mp::filter::SessionShared) @@ -872,7 +989,7 @@ Figure out what additional information we need in:
-
+
<literal>sru_z3950</literal> (mp::filter::SRUtoZ3950) @@ -894,7 +1011,7 @@ Figure out what additional information we need in:
-
+
<literal>template</literal> (mp::filter::Template) @@ -907,7 +1024,7 @@ Figure out what additional information we need in:
-
+
<literal>virt_db</literal> (mp::filter::VirtualDB) @@ -922,7 +1039,7 @@ Figure out what additional information we need in:
-
+
<literal>z3950_client</literal> (mp::filter::Z3950Client) @@ -940,7 +1057,7 @@ Figure out what additional information we need in:
-
+
<literal>zeerex_explain</literal> (mp::filter::ZeerexExplain) @@ -1008,7 +1125,7 @@ Figure out what additional information we need in: Configuration: the Metaproxy configuration file format -
+
Introductory notes If Metaproxy is an interpreter providing operations on packages, then @@ -1033,11 +1150,19 @@ Figure out what additional information we need in: <metaproxy xmlns="http://indexdata.com/metaproxy" version="1.0"> - The top-level element is <metaproxy>. This contains a - <start> element, a <filters> element and a - <routes> element, in that order. <filters> is - optional; the other two are mandatory. All three are - non-repeatable. + The top-level element is <metaproxy>. This contains + a <dlpath> element, + a <start> element, + a <filters> element and + a <routes> element, in that order. <dlpath> and + <filters> are optional; the other two are mandatory. + All four are non-repeatable. + + + The <dlpath;> element contains a text element which + specifies the location of filter modules. This is only needed + if Metaproxy must load 3rd party filters (most filters with Metaproxy + are built into the Metaproxy application). The <start> element is empty, but carries a @@ -1090,6 +1215,7 @@ Figure out what additional information we need in: + /usr/lib/metaproxy/modules @@ -1190,7 +1316,7 @@ Figure out what additional information we need in: Virtual databases and multi-database searching -
+
Introductory notes Two of Metaproxy's filters are concerned with multiple-database @@ -1524,13 +1650,8 @@ Z> merges them into a single Search response, which is what eventually makes it back to the client. -
- -
- A picture is worth a thousand words (but only five hundred on 64-bit architectures) - - + @@ -1547,13 +1668,10 @@ Z> document.] - - - +
@@ -1563,7 +1681,6 @@ Z> Metaproxy can act as SRU and - SRW web service server, which translates web service requests to ANSI/NISO Z39.50 packages and sends them off to common available targets. @@ -1654,9 +1771,8 @@ Z> - A typical minimal example SRU and - SRW server configuration file is found - in the tarball distribution at + A typical minimal example SRU + server configuration file is found in the tarball distribution at etc/config-sru-to-z3950.xml. @@ -1682,7 +1798,7 @@ Z> Classes in the Metaproxy source code -
+
Introductory notes Stop! Do not read this! @@ -1728,7 +1844,7 @@ Z> parentheses. -
+
<literal>mp::FactoryFilter</literal> (<filename>factory_filter.cpp</filename>) @@ -1743,7 +1859,7 @@ Z>
-
+
<literal>mp::FactoryStatic</literal> (<filename>factory_static.cpp</filename>) @@ -1758,7 +1874,7 @@ Z>
-
+
<literal>mp::filter::Base</literal> (<filename>filter.cpp</filename>) @@ -1775,7 +1891,7 @@ Z>
-
+
<literal>mp::filter::AuthSimple</literal>, <literal>Backend_test</literal>, etc. (<filename>filter_auth_simple.cpp</filename>, @@ -1817,7 +1933,7 @@ Z> </itemizedlist> </section> - <section> + <section id="class-Package"> <title><literal>mp::Package</literal> (<filename>package.cpp</filename>) @@ -1828,7 +1944,7 @@ Z>
-
+
<literal>mp::Pipe</literal> (<filename>pipe.cpp</filename>) @@ -1838,7 +1954,7 @@ Z>
-
+
<literal>mp::RouterChain</literal> (<filename>router_chain.cpp</filename>) @@ -1846,7 +1962,7 @@ Z>
-
+
<literal>mp::RouterFleXML</literal> (<filename>router_flexml.cpp</filename>) @@ -1854,7 +1970,7 @@ Z>
-
+
<literal>mp::Session</literal> (<filename>session.cpp</filename>) @@ -1862,7 +1978,7 @@ Z>
-
+
<literal>mp::ThreadPoolSocketObserver</literal> (<filename>thread_pool_observer.cpp</filename>) @@ -1870,7 +1986,7 @@ Z>
-
+
<literal>mp::util</literal> (<filename>util.cpp</filename>) @@ -1881,7 +1997,7 @@ Z>
-
+
<literal>mp::xml</literal> (<filename>xmlutil.cpp</filename>) @@ -1938,7 +2054,7 @@ Z> Reference - + The material in this chapter is drawn directly from the individual manual entries. In particular, the Metaproxy invocation section is @@ -1949,19 +2065,41 @@ Z> &manref; + + + License + + ©right; + + + Metaproxy is free software; you can redistribute it and/or modify it under + the terms of the GNU General Public License as published by the Free + Software Foundation; either version 2, or (at your option) any later + version. + + + + Metaproxy is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or + FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + for more details. + + + + You should have received a copy of the GNU General Public License + along with Metaproxy; see the file LICENSE. If not, write to the + Free Software Foundation, + 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + + + + + &gpl2; - +