+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
+ "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
+[
+ <!ENTITY % local SYSTEM "local.ent">
+ %local;
+ <!ENTITY % entities SYSTEM "entities.ent">
+ %entities;
+ <!ENTITY % idcommon SYSTEM "common/common.ent">
+ %idcommon;
+]>
+<book>
+ <bookinfo>
+ <title>YAZ User's Guide and Reference</title>
+ <authorgroup>
+ <author><firstname>Sebastian</firstname><surname>Hammer</surname></author>
+ <author><firstname>Adam</firstname><surname>Dickmeiss</surname></author>
+ <author><firstname>Mike</firstname><surname>Taylor</surname></author>
+ <author><firstname>Heikki</firstname><surname>Levanto</surname></author>
+ <author><firstname>Dennis</firstname><surname>Schafroth</surname></author>
+ </authorgroup>
+ <releaseinfo>&version;</releaseinfo>
+ <copyright>
+ <year>©right-year;</year>
+ <holder>Index Data</holder>
+ </copyright>
+ <abstract>
+ <simpara>
+ This document is the programmer's guide and reference to the &yaz;
+ package version &version;. &yaz; is a compact toolkit that provides
+ access to the Z39.50 and SRU/Solr protocols, as well as a set of
+ higher-level tools for implementing the server and client
+ roles, respectively.
+ The documentation can be used on its own, or as a reference when
+ looking at the example applications provided with the package.
+ </simpara>
+ <simpara>
+ <inlinemediaobject>
+ <imageobject>
+ <imagedata fileref="common/id.png" format="PNG"/>
+ </imageobject>
+ <imageobject>
+ <imagedata fileref="common/id.eps" format="EPS"/>
+ </imageobject>
+ </inlinemediaobject>
+ </simpara></abstract>
+ </bookinfo>
+ <chapter id="introduction">
+ <title>Introduction</title>
+ <para>
+ &yaz; is a C/C++ library for information retrieval applications
+ using the Z39.50/SRU/Solr protocols for information retrieval.
+ </para>
+ <para>
+ Properties of &yaz;:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Complete
+ <ulink url="&url.z39.50;">Z39.50</ulink> version 3 support.
+ Amendments and Z39.50-2002 revision is supported.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Supports
+ <ulink url="&url.sru;">SRU GET/POST/SOAP</ulink>
+ version 1.1, 1.2 and 2.0 (over HTTP and HTTPS).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Includes BER encoders/decoders for the
+ <ulink url="&url.ill;">ISO ILL</ulink>
+ protocol.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Supports
+ <ulink url="&url.solr;">Solr</ulink> Web Service version 1.4.x
+ (client side only)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Supports the following transports: BER over TCP/IP
+ (<ulink url="&url.ber.over.tcpip;">RFC1729</ulink>),
+ BER over unix local socket, and
+ <ulink url="&url.http.1.1;">HTTP 1.1</ulink>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Secure Socket Layer support using
+ <ulink url="&url.gnutls;">GnuTLS</ulink>.
+ If enabled, &yaz; uses HTTPS transport (for SOAP) or
+ "Secure BER" (for Z39.50).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Offers
+ <ulink url="&url.zoom;">ZOOM</ulink> C API implementing
+ Z39.50, SRU and Solr Web Service.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The &yaz; library offers a set of useful utilities
+ related to the protocols, such as MARC (ISO2709) parser,
+ CCL (ISO8777) parser,
+ <ulink url="&url.cql;">CQL</ulink>
+ parser, memory management routines, character set conversion.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Portable code. &yaz; compiles out-of-the box on most Unixes and
+ on Windows using Microsoft Visual C++.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fast operation. The C based BER encoders/decoders as well
+ as the server component of &yaz; is very fast.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Liberal license that allows for commercial use of &yaz;.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <sect1 id="introduction.reading">
+ <title>Reading this Manual</title>
+ <para>
+ Most implementors only need to read a fraction of the
+ material in thie manual, so a quick walkthrough of the chapters
+ is in order.
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <xref linkend="installation"/> contains installation
+ instructions for &yaz;. You don't need reading this
+ if you expect to download &yaz; binaries.
+ However, the chapter contains information about how
+ to make <emphasis>your</emphasis> application link
+ with &yaz;.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="zoom"/> describes the ZOOM API of &yaz;.
+ This is definitely worth a read if you wish to develop a Z39.50/SRU
+ client.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="server"/> describes the generic frontend server
+ and explains how to develop server Z39.50/SRU applications for &yaz;.
+ Obviously worth reading if you're to develop a server.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="yaz-client"/> describes how to use the &yaz; Z39.50
+ client. If you're developer and wish to test your server
+ or a server from another party, you might find this chapter
+ useful.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="asn"/> documents the most commonly used Z39.50
+ C data structures offered by the &yaz; API. Client
+ developers using ZOOM and non-Z39.50 implementors may skip this.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="soap"/> describes how SRU and SOAP is used
+ in &yaz;. Only if you're developing SRU applications
+ this section is a must.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="tools"/> contains sections for the various
+ tools offered by &yaz;. Scan through the material quickly
+ and see what's relevant to you! SRU implementors
+ might find the <link linkend="cql">CQL</link> section
+ particularly useful.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="odr"/> goes through the details of the
+ ODR module which is the work horse that encodes and decodes
+ BER packages. Implementors using ZOOM only, do <emphasis>not</emphasis>
+ need reading this.
+ Most other Z39.50 implementors only need to read the first two
+ sections (<xref linkend="odr.introduction"/> and
+ <xref linkend="odr.use"/>).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="comstack"/> describes the network layer module
+ COMSTACK. Implementors using ZOOM or the generic frontend server
+ may skip this. Others, presumably, handling client/server
+ communication on their own should read this.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </sect1>
+ <sect1 id="introduction.api">
+ <title>The API</title>
+ <para>
+ The <ulink url="&url.yaz;">&yaz;</ulink>
+ toolkit offers several different levels of access to the
+ <ulink url="&url.z39.50;">ISO23950/Z39.50</ulink>,
+ <ulink url="&url.ill;">ILL</ulink> and
+ <ulink url="&url.sru;">SRU</ulink>
+ protocols.
+ The level that you need to use depends on your requirements, and
+ the role (server or client) that you want to implement.
+ If you're developing a client application you should consider the
+ <link linkend="zoom">ZOOM</link> API.
+ It is, by far, the easiest way to develop clients in C.
+ Server implementers should consider the
+ <link linkend="server">generic frontend server</link>.
+ None of those high-level APIs support the whole protocol, but
+ they do include most facilities used in existing Z39.50 applications.
+ </para>
+ <para>
+ If you're using 'exotic' functionality (meaning anything not included in
+ the high-level APIs), developing non-standard extensions to Z39.50 or
+ you're going to develop an ILL application you'll have to learn the lower
+ level APIs of &yaz;.
+ </para>
+ <para>
+ The YAZ toolkit modules is shown in figure <xref linkend="yaz.layer"/>.
+ </para>
+ <figure id="yaz.layer">
+ <title>YAZ layers</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="apilayer.png" format="PNG"/>
+ </imageobject>
+ <imageobject>
+ <imagedata fileref="apilayer.eps" format="EPS"/>
+ </imageobject>
+ </mediaobject>
+ </figure>
+ <para>
+ There are four layers.
+ <itemizedlist>
+ <listitem>
+ <para>A client or server application (or both).
+ This layer includes ZOOM and the generic frontend server.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The second layer provides a C represenation of the
+ protocol units (packages) for Z39.50 ASN.1, ILL ASN.1,
+ SRU.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The third layer encodes and decodes protocol data units to
+ simple packages (buffer with certain length). The &odr; module
+ encodes and decodes BER whereas the HTTP modules encodes and
+ decodes HTTP ruquests/responses.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The lowest layer is &comstack; which exchanges the encoded packages
+ with a peer process over a network.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The &asn; module represents the ASN.1 definition of
+ the Z39.50 protocol. It establishes a set of type and
+ structure definitions, with one structure for each of the top-level
+ PDUs, and one structure or type for each of the contained ASN.1 types.
+ For primitive types, or other types that are defined by the ASN.1
+ standard itself (such as the EXTERNAL type), the C representation is
+ provided by the &odr; (Open Data Representation) subsystem.
+ </para>
+ <para>
+ &odr; is a basic mechanism for representing an
+ ASN.1 type in the C programming language, and for implementing BER
+ encoders and decoders for values of that type. The types defined in
+ the &asn; module generally have the prefix <literal>Z_</literal>, and
+ a suffix corresponding to the name of the type in the ASN.1
+ specification of the protocol (generally Z39.50-1995). In the case of
+ base types (those originating in the ASN.1 standard itself), the prefix
+ <literal>Odr_</literal> is sometimes seen. Either way, look for
+ the actual definition in either <filename>z-core.h</filename> (for the types
+ from the protocol), <filename>odr.h</filename> (for the primitive ASN.1
+ types).
+ The &asn; library also provides functions (which are, in turn,
+ defined using &odr; primitives) for encoding and decoding data values.
+ Their general form is
+ <funcsynopsis>
+ <funcprototype><funcdef>int <function>z_<replaceable>xxx</replaceable></function></funcdef>
+ <paramdef>ODR <parameter>o</parameter></paramdef>
+ <paramdef>Z_<replaceable>xxx</replaceable> **<parameter>p</parameter></paramdef>
+ <paramdef>int <parameter>optional</parameter></paramdef>
+ <paramdef>const char *<parameter>name</parameter></paramdef>
+ </funcprototype>
+ </funcsynopsis>
+ (note the lower-case "z" in the function name)
+ </para>
+ <note>
+ <para>
+ If you are using the premade definitions of the &asn; module, and you
+ are not adding new protocol of your own, the only parts of &odr; that you
+ need to worry about are documented in
+ <xref linkend="odr.use"/>.
+ </para>
+ </note>
+ <para>
+ When you have created a BER-encoded buffer, you can use the &comstack;
+ subsystem to transmit (or receive) data over the network. The &comstack;
+ module provides simple functions for establishing a connection
+ (passively or actively, depending on the role of your application),
+ and for exchanging BER-encoded PDUs over that connection. When you
+ create a connection endpoint, you need to specify what transport to
+ use (TCP/IP, SSL or UNIX sockets).
+ For the remainder of the connection's lifetime, you don't have
+ to worry about the underlying transport protocol at all - the &comstack;
+ will ensure that the correct mechanism is used.
+ </para>
+ <para>
+ We call the combined interfaces to &odr;, &asn;, and &comstack; the service
+ level API. It's the API that most closely models the Z39.50
+ service/protocol definition, and it provides unlimited access to all
+ fields and facilities of the protocol definitions.
+ </para>
+ <para>
+ The reason that the &yaz; service-level API is a conglomerate of the
+ APIs from three different submodules is twofold. First, we wanted to allow
+ the user a choice of different options for each major task. For instance,
+ if you don't like the protocol API provided by &odr;/&asn;, you
+ can use SNACC or BERUtils instead, and still have the benefits of the
+ transparent transport approach of the &comstack; module. Secondly,
+ we realize that you may have to fit the toolkit into an existing
+ event-processing structure, in a way that is incompatible with
+ the &comstack; interface or some other part of &yaz;.
+ </para>
+ </sect1>
+ </chapter>
+ <chapter id="installation">
+ <title>Compilation and Installation</title>
+ <sect1 id="installation-introduction">
+ <title>Introduction</title>
+ <para>
+ The latest version of the software will generally be found at:
+ </para>
+ <para>
+ <ulink url="&url.yaz.download;"/>
+ </para>
+ <para>
+ We have tried our best to keep the software portable, and on many
+ platforms, you should be able to compile everything with little or
+ no changes.
+ </para>
+ <para>
+ The software is regularly tested on
+ <ulink url="&url.debian;">Debian GNU/Linux</ulink>,
+ <ulink url="&url.centos;">CentOS</ulink>,
+ <ulink url="&url.ubuntu;">Ubuntu Linux</ulink>,
+ <ulink url="&url.freebsd;">FreeBSD (i386)</ulink>,
+ <ulink url="&url.macosx;">MAC OSX</ulink>,
+ <ulink url="&url.solaris;">Solaris</ulink>,
+ Windows 7, Windows XP.
+ </para>
+ <para>
+ Some versions have be known to work on HP/UX,
+ DEC Unix, <ulink url="&url.netbsd;">NetBSD</ulink>,
+ <ulink url="&url.openbsd;">OpenBSD</ulink>,
+ IBM AIX,
+ Data General DG/UX (with some CFLAGS tinkering),
+ SGI/IRIX, DDE Supermax, Apple Macintosh (using the Codewarrior programming
+ environment and the GUSI socket libraries),
+ IBM AS/400 .
+ </para>
+ <para>
+ If you move the software to other platforms, we'd be grateful if you'd
+ let us know about it. If you run into difficulties, we will try to help
+ if we can, and if you solve the problems, we would be happy to include
+ your fixes in the next release. So far, we have mostly avoided
+ <literal>#ifdefs</literal> for individual platforms, and we'd
+ like to keep it that way as far as it makes sense.
+ </para>
+ <para>
+ We maintain a mailing-list for the purpose of announcing new releases and
+ bug-fixes, as well as general discussion. Subscribe by
+ filling-in the form
+ <ulink url="&url.yaz.mailinglist;">here</ulink>.
+ General questions and problems can be directed at
+ <ulink url="&url.yaz.mail;"/>, or the address given at the top of
+ this document.
+ </para>
+ </sect1>
+ <sect1 id="installation.unix"><title>UNIX</title>
+ <para>
+ We provide
+ <ulink url="&url.debian;">Debian GNU/Linux</ulink> (i386 and amd64),
+ <ulink url="&url.ubuntu;">Ubuntu</ulink> (i386 and amd64)
+ and
+ <ulink url="&url.centos;">CentOS</ulink> (amd64 only) packages for &yaz;.
+ You should be able to create packages for other CPUs by building
+ them from the source package.
+ </para>
+ <para>
+ YAZ is also part of several packages repositories. Some of them are
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Solaris CSW: <ulink url="http://www.opencsw.org/packages/yaz/"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Solaris: <ulink url="http://unixpackages.com"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ FreeBSD: <ulink url="http://www.freshports.org/net/yaz"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Debian: <ulink url="http://packages.debian.org/search?keywords=yaz"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ubuntu: <ulink url="https://launchpad.net/ubuntu/+source/yaz"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ NetBSD:
+ <ulink url="http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/yaz/README.html"/>
+ </para>
+ </listitem>
+ </itemizedlist>
+ <sect2 id="installation.source.unix">
+ <title>Compiling from source on Unix</title>
+ <para>
+ Note that if your system doesn't have a native ANSI C compiler, you may
+ have to acquire one separately. We recommend
+ <ulink url="&url.gcc;">GCC</ulink>.
+ </para>
+ <para>
+ If you wish to use character set conversion facilities in &yaz; or if you
+ are compiling &yaz; for use with Zebra it is a good idea to ensure that
+ the iconv library is installed. Some Unixes today already have it
+ - if not, we suggest
+ <ulink url="&url.libiconv;">GNU libiconv</ulink>.
+ </para>
+ <para>
+ YAZ 3.0.16 and later includes a wrapper for the
+ <ulink url="&url.icu;">ICU</ulink>
+ (International Components for Unicode).
+ In order to use this, the developer version of the ICU library
+ must be available. ICU support is recommended for applications
+ such as Pazpar2 and Zebra.
+ </para>
+ <para>
+ The <ulink url="&url.libxslt;">libxslt</ulink>,
+ <ulink url="&url.libxml2;">libxml2</ulink> librararies are required
+ if &yaz; is to support SRU/Solr.
+ These libraries are very portable and should compile out-of-the
+ box on virtually all Unix platforms. It is available in binary
+ forms for Linux and others.
+ </para>
+ <para>
+ The GNU tools
+ <ulink url="&url.autoconf;">Autoconf</ulink>,
+ <ulink url="&url.automake;">Automake</ulink> and
+ <ulink url="&url.libtool;">Libtool</ulink>
+ are used to generate Makefiles and configure &yaz; for the system.
+ You do <emphasis>not</emphasis> these tools unless you're using the
+ Git version of &yaz;.
+ </para>
+ <para>
+ The CQL parser for &yaz; is built using
+ GNU <ulink url="&url.bison;">Bison</ulink>.
+ This tool is only needed if you're using the Git version of &yaz;.
+ </para>
+ <para>
+ &yaz; includes a tiny ASN.1 compiler. This compiler is
+ written in <ulink url="&url.tcl;">Tcl</ulink>.
+ But as for Bison you do not need it unless you're using Git
+ version of &yaz; or you're using the compiler to built own codecs
+ for private ASN.1.
+ </para>
+ <para>
+ Generally it should be sufficient to run configure without options,
+ like this:
+ </para>
+ <screen>
+ ./configure
+ </screen>
+ <para>
+ The configure script attempts to use use the C compiler specified by
+ the <literal>CC</literal> environment variable. If not set, GNU C will be
+ used if it is available. The <literal>CFLAGS</literal> environment
+ variable holds options to be passed to the C compiler. If you're using
+ Bourne-compatible shell you may pass something like this to use a
+ particular C compiler with optimization enabled:
+ </para>
+ <screen>
+ CC=/opt/ccs/bin/cc CFLAGS=-O ./configure
+ </screen>
+ <para>
+ To customize &yaz;, the configure script also accepts a set of options.
+ The most important are:
+ <variablelist>
+ <varlistentry>
+ <term>
+ <literal>--prefix</literal>=<replaceable>prefix</replaceable>
+ </term>
+ <listitem>
+ <para>Specifies installation prefix for &yaz;. This is
+ only needed if you run <literal>make install</literal> later to
+ perform a "system" installation. The prefix is
+ <literal>/usr/local</literal> if not specified.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--enable-tcpd</literal>
+ </term>
+ <listitem>
+ <para>The front end server will be built using Wietse's
+ <ulink url="&url.tcpwrapper;">TCP wrapper library</ulink>.
+ It allows you to allow/deny clients depending on IP number.
+ The TCP wrapper library is often used in GNU/Linux and
+ BSD distributions.
+ See
+ <citerefentry>
+ <refentrytitle>hosts_access</refentrytitle>
+ <manvolnum>5</manvolnum>
+ </citerefentry>
+ and
+ <citerefentry>
+ <refentrytitle>tcpd</refentrytitle>
+ <manvolnum>8</manvolnum>
+ </citerefentry>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--enable-threads</literal>
+ </term>
+ <listitem>
+ <para>&yaz; will be built using POSIX threads.
+ Specifically, <constant>_REENTRANT</constant> will be defined during
+ compilation.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--disable-shared</literal>
+ </term>
+ <listitem>
+ <para>The make process will not create shared
+ libraries (also known as shared objects <filename>.so</filename>).
+ By default, shared libraries are created -
+ equivalent to <literal>--enable-shared</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--disable-shared</literal>
+ </term>
+ <listitem>
+ <para>The make process will not create
+ static libraries (<filename>.a</filename>).
+ By default, static libraries are created -
+ equivalent to <literal>--enable-static</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-iconv</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>Compile &yaz; with iconv library in directory
+ <replaceable>prefix</replaceable>. By default configure will
+ search for iconv on the system. Use this option if it
+ doesn't find iconv. Alternatively,
+ <literal>--without-iconv</literal>, can be uset to force &yaz;
+ not to use iconv.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-xslt</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>Compile &yaz; with
+ <ulink url="&url.libxslt;">libxslt</ulink> in directory
+ <replaceable>prefix</replaceable>.
+ Use this option if you want XSLT and XML support.
+ By default, configure will
+ search for libxslt on the system. Use this option if it
+ libxslt is not found automatically. Alternatively,
+ <literal>--without-xslt</literal>, can be used to force &yaz;
+ not to use libxslt.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-xml2</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>Compile &yaz; with
+ <ulink url="&url.libxml2;">libxml2</ulink> in directory
+ <replaceable>prefix</replaceable>.
+ Use this option if you want &yaz; to use XML and support SRU/Solr.
+ By default, configure will
+ search for libxml2 on the system. Use this option if it
+ libxml2 is not found automatically. Alternatively,
+ <literal>--without-xml2</literal>, can be used to force &yaz;
+ not to use libxml2.
+ </para>
+ <para>
+ Note that option <literal>--with-xslt</literal>
+ also enables libxml2.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-gnutls</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>&yaz; will be linked with the GNU TLS libraries and
+ an SSL COMSTACK will be provided. By default configure enables
+ SSL support for YAZ if the GNU TLS development libraries are found
+ on the system.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-icu</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>&yaz; will be linked the
+ <ulink url="&url.icu;">ICU</ulink> library in the prefix if given.
+ If prefix is not given, the libraries exposed by the script
+ <application>icu-config</application> will be used if found.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>--with-libgcrypt</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>&yaz; will be linked with
+ <ulink url="&url.libgcrypt;">Libgcrypt</ulink> in the prefix if given.
+ If prefix is not given, the libraries exposed by the script
+ <application>libgcrypt-config</application> will be used if found.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-memcached</literal>
+ </term>
+ <listitem>
+ <para>&yaz; will be linked with
+ <ulink url="&url.libmemcached;">libMemcached</ulink> to allow
+ for result-set caching for ZOOM.
+ The prefix can not be given. Note that YAZ will only search
+ for libMemcached if Libgcrypt is also enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ When configured, build the software by typing:
+ <screen>
+ make
+ </screen>
+ </para>
+ <para>
+ The following files are generated by the make process:
+ <variablelist>
+ <varlistentry>
+ <term><filename>src/libyaz.la</filename></term>
+ <listitem><para>
+ Main &yaz; library. This is no ordinary library. It's
+ a Libtool archive.
+ By default, &yaz; creates a static library in
+ <filename>lib/.libs/libyaz.a</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>src/libyaz_server.la</filename></term>
+ <listitem><para>
+ Generic Frontend server. This is an add-on for libyaz.la.
+ Code in this library uses POSIX threads functions - if POSIX
+ threads are available on the platform.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>src/libyaz_icu.la</filename></term>
+ <listitem><para>
+ Functions that wrap the ICU library.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>ztest/yaz-ztest</filename></term>
+ <listitem><para>Test Z39.50 server.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>client/yaz-client</filename></term>
+ <listitem><para>Z39.50 client for testing the protocol.
+ See chapter <link linkend="yaz-client">
+ YAZ client</link> for more information.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-config</filename></term>
+ <listitem><para>A Bourne-shell script, generated by configure, that
+ specifies how external applications should compile - and link with
+ &yaz;.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-asncomp</filename></term>
+ <listitem><para>The ASN.1 compiler for &yaz;. Requires the
+ Tcl Shell, <application>tclsh</application>, in
+ <literal>PATH</literal> to operate.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-iconv</filename></term>
+ <listitem><para>This program converts data in one character set to
+ another. This command exercises the YAZ character set
+ conversion API.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-marcdump</filename></term>
+ <listitem><para>This program parses ISO2709 encoded MARC records
+ and prints them in line-format or XML.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-icu</filename></term>
+ <listitem><para>This program exposes the ICU wrapper library if that
+ is enabled for YAZ. Only if ICU is available this program is
+ useful.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-url</filename></term>
+ <listitem><para>This program is a simple HTTP page fetcher ala
+ wget or curl.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>zoom/zoomsh</filename></term>
+ <listitem><para>
+ A simple shell implemented on top of the
+ <link linkend="zoom">ZOOM</link> functions.
+ The shell is a command line application that allows you to enter
+ simple commands to perform ZOOM operations.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>zoom/zoomtst1</filename>,
+ <filename>zoom/zoomtst2</filename>, ..</term>
+ <listitem><para>
+ Several small applications that demonstrates the ZOOM API.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ If you wish to install &yaz; in system directories
+ <filename>/usr/local/bin</filename>,
+ <filename>/usr/local/lib</filename> .. etc, you can type:
+ </para>
+ <screen>
+ make install
+ </screen>
+ <para>
+ You probably need to have root access in order to perform this.
+ You must specify the <literal>--prefix</literal> option for configure if
+ you wish to install &yaz; in other directories than the default
+ <filename>/usr/local/</filename>.
+ </para>
+ <para>
+ If you wish to perform an un-installation of &yaz;, use:
+ </para>
+ <screen>
+ make uninstall
+ </screen>
+ <para>
+ This will only work if you haven't reconfigured &yaz; (and therefore
+ changed installation prefix). Note that uninstall will not
+ remove directories created by make install, e.g.
+ <filename>/usr/local/include/yaz</filename>.
+ </para>
+ </sect2>
+ <sect2 id="installation-linking-yaz-unix">
+ <title>How to make apps using YAZ on UNIX</title>
+ <para>
+ This section describes how to compile - and link your own
+ applications using the &yaz; toolkit.
+ If you're used to Makefiles this shouldn't be hard. As for
+ other libraries you have used before, you have to set a proper include
+ path for your C/C++ compiler and specify the location of
+ &yaz; libraries. You can do it by hand, but generally we suggest
+ you use the <filename>yaz-config</filename> that is generated
+ by <filename>configure</filename>. This is especially
+ important if you're using the threaded version of &yaz; which
+ require you to pass more options to your linker/compiler.
+ </para>
+ <para>
+ The <filename>yaz-config</filename> script accepts command line
+ options that makes the <filename>yaz-config</filename> script print
+ options that you should use in your make process.
+ The most important ones are:
+ <literal>--cflags</literal>, <literal>--libs</literal>
+ which prints C compiler flags, and linker flags respectively.
+ </para>
+ <para>
+ A small and complete <literal>Makefile</literal> for a C
+ application consisting of one source file,
+ <filename>myprog.c</filename>, may look like this:
+ <screen>
+ YAZCONFIG=/usr/local/bin/yaz-config
+ CFLAGS=`$(YAZCONFIG) --cflags`
+ LIBS=`$(YAZCONFIG) --libs`
+ myprog: myprog.o
+ $(CC) $(CFLAGS) -o myprog myprog.o $(LIBS)
+ </screen>
+ </para>
+ <para>
+ The CFLAGS variable consists of a C compiler directive that will set
+ the include path to the <emphasis>parent</emphasis> directory
+ of <filename>yaz</filename>. That is, if &yaz; header files were
+ installed in <filename>/usr/local/include/yaz</filename>,
+ then include path is set to <filename>/usr/local/include</filename>.
+ Therefore, in your applications you should use
+ <screen>
+ #include <yaz/proto.h>
+ </screen>
+ and <emphasis>not</emphasis>
+ <screen>
+ #include <proto.h>
+ </screen>
+ </para>
+ <para>
+ For Libtool users, the <filename>yaz-config</filename> script provides
+ a different variant of option <literal>--libs</literal>, called
+ <literal>--lalibs</literal> that returns the name of the
+ Libtool archive(s) for &yaz; rather than the ordinary ones.
+ </para>
+ <para>
+ For applications using the threaded version of &yaz;,
+ specify <literal>threads</literal> after the
+ other options. When <literal>threads</literal> is given,
+ more flags and linker flags will be printed by
+ <filename>yaz-config</filename>. If our previous example was
+ using threads, you'd have to modify the lines that set
+ <literal>CFLAGS</literal> and <literal>LIBS</literal> as
+ follows:
+ <screen>
+ CFLAGS=`$(YAZCONFIG) --cflags threads`
+ LIBS=`$(YAZCONFIG) --libs threads`
+ </screen>
+ There is no need specify POSIX thread libraries in your Makefile.
+ The <literal>LIBS</literal> variable includes that as well.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="installation.win32">
+ <title>WIN32</title>
+ <para>The easiest way to install YAZ on Windows is by downloading
+ an installer from
+ <ulink url="&url.yaz.download.win32;">here</ulink>.
+ The installer comes with source too - in case you wish to
+ compile YAZ with different compiler options, etc.
+ </para>
+
+ <sect2 id="installation.win32.source">
+ <title>Compiling from Source on WIN32</title>
+ <para>
+ &yaz; is shipped with "makefiles" for the NMAKE tool that comes
+ with <ulink url="&url.vstudio;">
+ Microsoft Visual Studio</ulink>. It has been tested with
+ Microsoft Visual Studio 2003/2005/2008.
+ </para>
+ <para>
+ Start a command prompt and switch the sub directory
+ <filename>WIN</filename> where the file <filename>makefile</filename>
+ is located. Customize the installation by editing the
+ <filename>makefile</filename> file (for example by using notepad).
+ The following summarizes the most important settings in that file:
+ <variablelist>
+ <varlistentry>
+ <term><literal>DEBUG</literal></term>
+ <listitem><para>
+ If set to 1, the software is
+ compiled with debugging libraries (code generation is
+ multi-threaded debug DLL).
+ If set to 0, the software is compiled with release libraries
+ (code generation is multi-threaded DLL).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_TCL</literal>, <literal>TCL</literal></term>
+ <listitem><para>
+ If <literal>HAVE_TCL</literal> is set to 1, nmake will
+ use the ASN.1 compiler (<ulink url="&url.tcl;">Tcl</ulink> based).
+ You must set <literal>TCL</literal> to the full path of the Tcl
+ interpreter. A Windows version of Tcl is part of
+ <ulink url="&url.gitwindows;">Git for Windows</ulink>.
+ </para>
+ <para>
+ If you do not have Tcl installed, set
+ <literal>HAVE_TCL</literal> to 0.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_BISON</literal>,
+ <literal>BISON</literal></term>
+ <listitem><para>
+ If GNU Bison is present, you might set <literal>HAVE_BISON</literal>
+ to 1 and specify the Bison executable in <literal>BISON</literal>.
+ Bison is only required if you use the Git version of
+ YAZ or if you modify the grammar for CQL
+ (<filename>cql.y</filename>).
+ </para>
+ <para>
+ A Windows version of GNU Bison is part of
+ <ulink url="&url.gitwindows;">Git for Windows</ulink>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_ICONV</literal>,
+ <literal>ICONV_DIR</literal></term>
+ <listitem><para>
+ If <literal>HAVE_ICONV</literal> is set to 1, YAZ is compiled
+ with iconv support. In this configuration, set
+ <literal>ICONV_DIR</literal> to the iconv source directory.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_LIBXML2</literal>,
+ <literal>LIBXML2_DIR</literal></term>
+ <listitem>
+ <para>
+ If <literal>HAVE_LIBXML2</literal> is set to 1, YAZ is compiled
+ with SRU support. In this configuration, set
+ <literal>LIBXML2_DIR</literal> to the
+ <ulink url="&url.libxml2;">libxml2</ulink> source directory
+ and
+ <literal>ZLIB_DIR</literal> to the zlib directory.
+ </para>
+ <para>
+ Windows versions of libxslt, libxml2, zlib and iconv can be found
+ <ulink url="&url.libxml2.download.win32;">
+ Igor Zlatković' site</ulink>.
+ </para>
+ <note>
+ <para>
+ YAZ is not using zlib but libxml2 is depending on it.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_LIBXSLT</literal>,
+ <literal>LIBXSLT_DIR</literal></term>
+ <listitem>
+ <para>
+ If <literal>HAVE_LIBXSLT</literal> is set to 1, YAZ is compiled
+ with XSLT support. In this configuration, set
+ <literal>LIBXSLT_DIR</literal> to the
+ <ulink url="&url.libxslt;">libxslt</ulink> source directory.
+ </para>
+ <note>
+ <para>
+ libxslt depends libxml2.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_ICU</literal>,
+ <literal>ICU_DIR</literal></term>
+ <listitem>
+ <para>
+ If <literal>HAVE_ICU</literal> is set to 1, YAZ is compiled
+ with <ulink url="&url.icu;">ICU</ulink> support.
+ In this configuration, set
+ <literal>ICU_DIR</literal> to the
+ <ulink url="&url.icu;">ICU</ulink> source directory.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ When satisfied with the settings in the makefile, type
+ <screen>
+ nmake
+ </screen>
+ </para>
+ <note>
+ <para>
+ If the <filename>nmake</filename> command is not found on your system
+ you probably haven't defined the environment variables required to
+ use that tool. To fix that, find and run the batch file
+ <filename>vcvars32.bat</filename>. You need to run it from within
+ the command prompt or set the environment variables "globally";
+ otherwise it doesn't work.
+ </para>
+ </note>
+ <para>
+ If you wish to recompile &yaz; - for example if you modify
+ settings in the <filename>makefile</filename> you can delete
+ object files, etc by running.
+ <screen>
+ nmake clean
+ </screen>
+ </para>
+ <para>
+ The following files are generated upon successful compilation:
+ <variablelist>
+ <varlistentry>
+ <term><filename>bin/yaz&soversion;.dll</filename> /
+ <filename>bin/yaz&soversion;d.dll</filename></term>
+ <listitem><para>
+ &yaz; Release/Debug DLL.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>lib/yaz&soversion;.lib</filename> /
+ <filename>lib/yaz&soversion;d.lib</filename></term>
+ <listitem><para>
+ Import library for <filename>yaz&soversion;.dll</filename> /
+ <filename>yaz&soversion;d.dll</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz_cond&soversion;.dll</filename> /
+ <filename>bin/yaz_cond&soversion;d.dll</filename></term>
+ <listitem><para>
+ Release/Debug DLL for condition variable utilities (condvar.c).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>lib/yaz_cond&soversion;.lib</filename> /
+ <filename>lib/yaz_cond&soversion;d.lib</filename></term>
+ <listitem><para>
+ Import library for <filename>yaz_cond&soversion;.dll</filename> /
+ <filename>yaz_cond&soversion;d.dll</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz_icu&soversion;.dll</filename> /
+ <filename>bin/yaz_icu&soversion;d.dll</filename></term>
+ <listitem><para>
+ Release/Debug DLL for the ICU wrapper utility.
+ Only build if HAVE_ICU is 1.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>lib/yaz_icu&soversion;.lib</filename> /
+ <filename>lib/yaz_icu&soversion;d.lib</filename></term>
+ <listitem><para>
+ Import library for <filename>yaz_icu&soversion;.dll</filename> /
+ <filename>yaz_icu&soversion;d.dll</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz-ztest.exe</filename></term>
+ <listitem><para>
+ Z39.50 multi-threaded test/example server. It's a WIN32
+ console application.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz-client.exe</filename></term>
+ <listitem><para>
+ &yaz; Z39.50 client application. It's a WIN32 console application.
+ See chapter <link linkend="yaz-client">YAZ client</link> for more
+ information.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz-icu.exe</filename></term>
+ <listitem><para>This program exposes the ICU wrapper library if that
+ is enabled for YAZ. Only if ICU is available this program is
+ build.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/zoomsh.exe</filename></term>
+ <listitem><para>
+ Simple console application implemented on top of the
+ <link linkend="zoom">ZOOM</link> functions.
+ The application is a command line shell that allows you to enter
+ simple commands to perform ZOOM operations.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/zoomtst1.exe</filename>,
+ <filename>bin/zoomtst2.exe</filename>, ..</term>
+ <listitem><para>
+ Several small applications that demonstrates the ZOOM API.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+
+ <sect2 id="installation-linking-yaz-win32">
+ <title>How to make apps using YAZ on WIN32</title>
+ <para>
+ This section will go though the process of linking your WIN32
+ applications with &yaz;.
+ </para>
+ <para>
+ Some people are confused by the fact that we use the nmake
+ tool to build &yaz;. They think they have to do that too - in order
+ to make their WIN32 applications work with &yaz;. The good news is that
+ you don't have to. You can use the integrated environment of
+ Visual Studio if desired for your own application.
+ </para>
+ <para>
+ When setting up a project or Makefile you have to set the following:
+ <variablelist>
+ <varlistentry>
+ <term>include path</term>
+ <listitem><para>
+ Set it to the <filename>include</filename> directory of &yaz;.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>import library <filename>yaz&soversion;.lib</filename></term>
+ <listitem><para>
+ You must link with this library. It's located in the
+ sub directory <filename>lib</filename> of &yaz;.
+ If you want to link with the debug version of &yaz;, you must
+ link against <filename>yaz&soversion;d.lib</filename> instead.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>dynamic link library
+ <filename>yaz&soversion;.dll</filename>
+ </term>
+ <listitem><para>
+ This DLL must be in your execution path when you invoke
+ your application. Specifically, you should distribute this
+ DLL with your application.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+ </sect1>
+ </chapter>
+ <!--
+ ### Still to document:
+ ZOOM_connection_errcode(c)
+ ZOOM_connection_errmsg(c)
+ ZOOM_connection_addinfo(c)
+ ZOOM_connection_addinfo(c)
+ ZOOM_connection_diagset(c);
+ ZOOM_connection_save_apdu_wrbuf
+ ZOOM_diag_str(error)
+ ZOOM_resultset_record_immediate(s, pos)
+ ZOOM_resultset_cache_reset(r)
+ ZOOM_options_set_callback(opt, function, handle)
+ ZOOM_options_create_with_parent2(parent1, parent2)
+ ZOOM_options_getl(opt, name, len)
+ ZOOM_options_setl(opt, name, value, len)
+ ZOOM_options_get_bool(opt, name, defa)
+ ZOOM_options_get_int(opt, name, defa)
+ ZOOM_options_set_int(opt, name, value)
+ -->
+ <chapter id="zoom">
+ <title>ZOOM</title>
+ <para>
+ &zoom; is an acronym for 'Z39.50 Object-Orientation Model' and is
+ an initiative started by Mike Taylor (Mike is from the UK, which
+ explains the peculiar name of the model). The goal of &zoom; is to
+ provide a common Z39.50 client API not bound to a particular
+ programming language or toolkit.
+ </para>
+ <para>
+ From YAZ version 2.1.12, <ulink url="&url.sru;">SRU</ulink> is supported.
+ You can make SRU ZOOM connections by specifying scheme
+ <literal>http://</literal> for the hostname for a connection.
+ The dialect of SRU used is specified by the value of the
+ connection's <literal>sru</literal> option, which may be SRU over
+ HTTP GET (<literal>get</literal>),
+ SRU over HTTP POST (<literal>post</literal>), (SRU over
+ SOAP) (<literal>soap</literal>) or <literal>solr</literal>
+ (<ulink url="&url.solr;">Solr</ulink> Web Service).
+ Using the facility for embedding options in target strings, a
+ connection can be forced to use SRU rather the SRW (the default) by
+ prefixing the target string with <literal>sru=get,</literal>, like this:
+ <literal>sru=get,http://sru.miketaylor.org.uk:80/sru.pl</literal>
+ </para>
+ <para>
+ <ulink url="&url.solr;">Solr</ulink> protocol support was added to
+ YAZ in version 4.1.0, as a dialect of a SRU protocol, since both are
+ HTTP based protocols.
+ </para>
+ <para>
+ The lack of a simple Z39.50 client API for &yaz; has become more
+ and more apparent over time. So when the first &zoom; specification
+ became available,
+ an implementation for &yaz; was quickly developed. For the first time, it is
+ now as easy (or easier!) to develop clients than servers with &yaz;. This
+ chapter describes the &zoom; C binding. Before going further, please
+ reconsider whether C is the right programming language for the job.
+ There are other language bindings available for &yaz;, and still
+ more
+ are in active development. See the
+ <ulink url="&url.zoom;">ZOOM web-site</ulink> for
+ more information.
+ </para>
+ <para>
+ In order to fully understand this chapter you should read and
+ try the example programs <literal>zoomtst1.c</literal>,
+ <literal>zoomtst2.c</literal>, .. in the <literal>zoom</literal>
+ directory.
+ </para>
+ <para>
+ The C language misses features found in object oriented languages
+ such as C++, Java, etc. For example, you'll have to manually,
+ destroy all objects you create, even though you may think of them as
+ temporary. Most objects has a <literal>_create</literal> - and a
+ <literal>_destroy</literal> variant.
+ All objects are in fact pointers to internal stuff, but you don't see
+ that because of typedefs. All destroy methods should gracefully ignore a
+ <literal>NULL</literal> pointer.
+ </para>
+ <para>
+ In each of the sections below you'll find a sub section called
+ protocol behavior, that describes how the API maps to the Z39.50
+ protocol.
+ </para>
+ <sect1 id="zoom-connections">
+ <title>Connections</title>
+ <para>The Connection object is a session with a target.
+ </para>
+ <synopsis>
+ #include <yaz/zoom.h>
+
+ ZOOM_connection ZOOM_connection_new(const char *host, int portnum);
+
+ ZOOM_connection ZOOM_connection_create(ZOOM_options options);
+
+ void ZOOM_connection_connect(ZOOM_connection c, const char *host,
+ int portnum);
+ void ZOOM_connection_destroy(ZOOM_connection c);
+ </synopsis>
+ <para>
+ Connection objects are created with either function
+ <function>ZOOM_connection_new</function> or
+ <function>ZOOM_connection_create</function>.
+ The former creates and automatically attempts to establish a network
+ connection with the target. The latter doesn't establish
+ a connection immediately, thus allowing you to specify options
+ before establishing network connection using the function
+ <function>ZOOM_connection_connect</function>.
+ If the port number, <literal>portnum</literal>, is zero, the
+ <literal>host</literal> is consulted for a port specification.
+ If no port is given, 210 is used. A colon denotes the beginning of
+ a port number in the host string. If the host string includes a
+ slash, the following part specifies a database for the connection.
+ </para>
+ <para>
+ You can prefix the host with a scheme followed by colon. The
+ default scheme is <literal>tcp</literal> (Z39.50 protocol).
+ The scheme <literal>http</literal> selects SRU/get over HTTP by default,
+ but can overridded to use SRU/post, SRW and the Solr protocol.
+ </para>
+ <para>
+ You can prefix the scheme-qualified host-string with one or more
+ comma-separated
+ <literal><parameter>key</parameter>=<parameter>value</parameter></literal>
+ sequences, each of which represents an option to be set into the
+ connection structure <emphasis>before</emphasis> the
+ protocol-level connection is forged and the initialization
+ handshake takes place. This facility can be used to provide
+ authentication credentials, as in host-strings such as:
+ <literal>user=admin,password=halfAm4n,tcp:localhost:8017/db</literal>
+ </para>
+ <para>
+ Connection objects should be destroyed using the function
+ <function>ZOOM_connection_destroy</function>.
+ </para>
+ <synopsis>
+ void ZOOM_connection_option_set(ZOOM_connection c,
+ const char *key, const char *val);
+
+ void ZOOM_connection_option_setl(ZOOM_connection c,
+ const char *key,
+ const char *val, int len);
+
+ const char *ZOOM_connection_option_get(ZOOM_connection c,
+ const char *key);
+ const char *ZOOM_connection_option_getl(ZOOM_connection c,
+ const char *key,
+ int *lenp);
+ </synopsis>
+ <para>
+ The functions <function>ZOOM_connection_option_set</function> and
+ <function>ZOOM_connection_option_setl</function> allows you to
+ set an option given by <parameter>key</parameter> to the value
+ <parameter>value</parameter> for the connection.
+ For <function>ZOOM_connection_option_set</function>, the
+ value is assumed to be a 0-terminated string. Function
+ <function>ZOOM_connection_option_setl</function> specifies a
+ value of a certain size (len).
+ </para>
+ <para>
+ Functions <function>ZOOM_connection_option_get</function> and
+ <function>ZOOM_connection_option_getl</function> returns
+ the value for an option given by <parameter>key</parameter>.
+ </para>
+ <table id="zoom-connection-options" frame="top">
+ <title>ZOOM Connection Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ implementationName</entry><entry>Name of Your client
+ </entry><entry>none</entry></row>
+ <row><entry>
+ user</entry><entry>Authentication user name
+ </entry><entry>none</entry></row>
+ <row><entry>
+ group</entry><entry>Authentication group name
+ </entry><entry>none</entry></row>
+ <row><entry>
+ password</entry><entry>Authentication password.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ authenticationMode</entry><entry>How authentication is encoded.
+ </entry><entry>basic</entry></row>
+ <row><entry>
+ host</entry><entry>Target host. This setting is "read-only".
+ It's automatically set internally when connecting to a target.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ proxy</entry><entry>Proxy host. If set, the logical host
+ is encoded in the otherInfo area of the Z39.50 Init PDU
+ with OID 1.2.840.10003.10.1000.81.1.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ clientIP</entry><entry>Client IP. If set, is
+ encoded in the otherInfo area of a Z39.50 PDU with OID
+ 1.2.840.10003.10.1000.81.3. Holds the original IP addreses
+ of a client. Is used of ZOOM is used in a gateway of some sort.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ async</entry><entry>If true (1) the connection operates in
+ asynchronous operation which means that all calls are non-blocking
+ except
+ <link linkend="zoom.events"><function>ZOOM_event</function></link>.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ maximumRecordSize</entry><entry> Maximum size of single record.
+ </entry><entry>1 MB</entry></row>
+ <row><entry>
+ preferredMessageSize</entry><entry> Maximum size of multiple records.
+ </entry><entry>1 MB</entry></row>
+ <row><entry>
+ lang</entry><entry> Language for negotiation.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ charset</entry><entry> Character set for negotiation.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ serverImplementationId</entry><entry>
+ Implementation ID of server. (The old targetImplementationId
+ option is also supported for the benefit of old applications.)
+ </entry><entry>none</entry></row>
+ <row><entry>
+ targetImplementationName</entry><entry>
+ Implementation Name of server. (The old
+ targetImplementationName option is also supported for the
+ benefit of old applications.)
+ </entry><entry>none</entry></row>
+ <row><entry>
+ serverImplementationVersion</entry><entry>
+ Implementation Version of server. (the old
+ targetImplementationVersion option is also supported for the
+ benefit of old applications.)
+ </entry><entry>none</entry></row>
+ <row><entry>
+ databaseName</entry><entry>One or more database names
+ separated by character plus (<literal>+</literal>), which to
+ be used by subsequent search requests on this Connection.
+ </entry><entry>Default</entry></row>
+ <row><entry>
+ piggyback</entry><entry>True (1) if piggyback should be
+ used in searches; false (0) if not.
+ </entry><entry>1</entry></row>
+ <row><entry>
+ smallSetUpperBound</entry><entry>If hits is less than or equal to this
+ value, then target will return all records using small element set name
+ </entry><entry>0</entry></row>
+ <row><entry>
+ largeSetLowerBound</entry><entry>If hits is greater than this
+ value, the target will return no records.
+ </entry><entry>1</entry></row>
+ <row><entry>
+ mediumSetPresentNumber</entry><entry>This value represents
+ the number of records to be returned as part of a search when when
+ hits is less than or equal to large set lower bound and if hits
+ is greater than small set upper bound.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ smallSetElementSetName</entry><entry>
+ The element set name to be used for small result sets.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ mediumSetElementSetName</entry><entry>
+ The element set name to be for medium-sized result sets.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ init_opt_search, init_opt_present, init_opt_delSet, etc.</entry><entry>
+ After a successful Init, these options may be interrogated to
+ discover whether the server claims to support the specified
+ operations.
+ </entry><entry>none</entry></row>
+ <row>
+ <entry>sru</entry><entry>
+ SRU/Solr transport type. Must be either <literal>soap</literal>,
+ <literal>get</literal>, <literal>post</literal>, or
+ <literal>solr</literal>.
+ </entry><entry>soap</entry></row>
+ <row><entry>
+ sru_version</entry><entry>
+ SRU/SRW version. Should be <literal>1.1</literal>, or
+ <literal>1.2</literal>. This is , prior to connect, the version
+ to offer (highest version). And following connect (in fact
+ first operation), holds the negotiated version with the server
+ (same or lower version).
+ </entry><entry>1.2</entry></row>
+ <row id="zoom.facets.option"><entry>
+ facets</entry><entry>
+ Requested or recommend facets may be given before a search is sent.
+ The value of this setting is described in <xref linkend="facets"/>
+ For inspection of the facets returned, refer to the functions
+ described in <xref linkend="zoom.facets"/>.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ apdulog</entry><entry>
+ If set to a true value such as "1", a log of low-level
+ protocol packets is emitted on standard error stream. This
+ can be very useful for debugging.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ saveAPDU</entry><entry>
+ If set to a true value such as "1", a log of low-level
+ protocol packets is saved. The log can be retrieved by reading
+ option APDU. Setting saveAPDU always has the side effect of
+ resetting the currently saved log. This setting is
+ <emphasis>write-only</emphasis>. If read, NULL will be returned.
+ It is only recognized in
+ <function>ZOOM_connection_option_set</function>.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ APDU</entry><entry>
+ Returns the log of protocol packets. Will be empty if logging
+ is not enabled (see saveAPDU above). This setting is
+ <emphasis>read-only</emphasis>. It is only recognized if used
+ in call to <function>ZOOM_connection_option_get</function> or
+ <function>ZOOM_connection_option_getl</function>.
+ </entry><entry></entry></row>
+ <row><entry>
+ memcached</entry><entry>
+ If given and non-empty,
+ <ulink url="&url.libmemcached;">libMemcached</ulink>
+ will be configured for the connection.
+ This option is inspected by ZOOM when a connection is established.
+ If the <literal>memcached</literal> option is given
+ and YAZ is compiled without libMemcached support, an internal
+ diagnostic (10018) will be thrown.
+ libMemcached support is available for YAZ 5.0.13 or later. If this
+ option is supplied for an earlier version of YAZ, it is
+ <emphasis>ignored</emphasis>.
+ The value of this option is a string passed verbatim to
+ the <function>memcached</function> function part of libMemcached.
+ Refer to man page
+ <ulink url="http://manned.org/memcached.3">memcached(3)</ulink>.
+ Earlier versions of libMemcached
+ do not offer this function. In this case only the option
+ <literal>--server=</literal><replaceable>host</replaceable> may
+ be given (YAZ emulates that part of libMemcached).
+ </entry><entry>none</entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ If either option <literal>lang</literal> or <literal>charset</literal>
+ is set, then
+ <ulink url="&url.z39.50.charneg;">
+ Character Set and Language Negotiation</ulink> is in effect.
+ </para>
+ <synopsis>
+ int ZOOM_connection_error(ZOOM_connection c, const char **cp,
+ const char **addinfo);
+ int ZOOM_connection_error_x(ZOOM_connection c, const char **cp,
+ const char **addinfo, const char **dset);
+ </synopsis>
+ <para>
+ Function <function>ZOOM_connection_error</function> checks for
+ errors for the last operation(s) performed. The function returns
+ zero if no errors occurred; non-zero otherwise indicating the error.
+ Pointers <parameter>cp</parameter> and <parameter>addinfo</parameter>
+ holds messages for the error and additional-info if passed as
+ non-<literal>NULL</literal>. Function
+ <function>ZOOM_connection_error_x</function> is an extended version
+ of <function>ZOOM_connection_error</function> that is capable of
+ returning name of diagnostic set in <parameter>dset</parameter>.
+ </para>
+ <sect2 id="zoom-connection-z39.50">
+ <title>Z39.50 Protocol behavior</title>
+ <para>
+ The calls <function>ZOOM_connection_new</function> and
+ <function>ZOOM_connection_connect</function> establishes a TCP/IP
+ connection and sends an Initialize Request to the target if
+ possible. In addition, the calls waits for an Initialize Response
+ from the target and the result is inspected (OK or rejected).
+ </para>
+ <para>
+ If <literal>proxy</literal> is set then the client will establish
+ a TCP/IP connection with the peer as specified by the
+ <literal>proxy</literal> host and the hostname as part of the
+ connect calls will be set as part of the Initialize Request.
+ The proxy server will then "forward" the PDU's transparently
+ to the target behind the proxy.
+ </para>
+ <para>
+ For the authentication parameters, if option <literal>user</literal>
+ is set and both options <literal>group</literal> and
+ <literal>pass</literal> are unset, then Open style
+ authentication is used (Version 2/3) in which case the username
+ is usually followed by a slash, then by a password.
+ If either <literal>group</literal>
+ or <literal>pass</literal> is set then idPass authentication
+ (Version 3 only) is used. If none of the options are set, no
+ authentication parameters are set as part of the Initialize Request
+ (obviously).
+ </para>
+ <para>
+ When option <literal>async</literal> is 1, it really means that
+ all network operations are postponed (and queued) until the
+ function <literal>ZOOM_event</literal> is invoked. When doing so
+ it doesn't make sense to check for errors after
+ <literal>ZOOM_connection_new</literal> is called since that
+ operation "connecting - and init" is still incomplete and the
+ API cannot tell the outcome (yet).
+ </para>
+ </sect2>
+ <sect2 id="zoom.sru.init.behavior">
+ <title>SRU/Solr Protocol behavior</title>
+ <para>
+ The HTTP based protocols (SRU, SRW, Solr) doesn't feature an
+ Inititialize Request, so the connection phase merely establishes a
+ TCP/IP connection with the HTTP server.
+ </para>
+ <para>Most of the ZOOM connection options do not
+ affect SRU/Solr and they are ignored. However, future versions
+ of &yaz; might honor <literal>implementationName</literal> and
+ put that as part of User-Agent header for HTTP requests.
+ </para>
+ <para>
+ The <literal>charset</literal> is used in the Content-Type header
+ of HTTP requests.
+ </para>
+ <para>
+ Setting <literal>authentcationMode</literal> specifies how
+ authentication parameters are encoded for HTTP. The default is
+ "<literal>basic</literal>" where <literal>user</literal> and
+ <literal>password</literal> are encoded by using HTTP basic
+ authentication.
+ </para>
+ <para>
+ If <literal>authentcationMode</literal> is "<literal>url</literal>", then
+ user and password are encoded in the URL by parameters
+ <literal>x-username</literal> and <literal>x-password</literal> as
+ given by the SRU standard.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.query">
+ <title>Queries</title>
+ <para>
+ Query objects represents queries.
+ </para>
+ <synopsis>
+ ZOOM_query ZOOM_query_create(void);
+
+ void ZOOM_query_destroy(ZOOM_query q);
+
+ int ZOOM_query_prefix(ZOOM_query q, const char *str);
+
+ int ZOOM_query_cql(ZOOM_query s, const char *str);
+
+ int ZOOM_query_sortby(ZOOM_query q, const char *criteria);
+
+ int ZOOM_query_sortby2(ZOOM_query q, const char *strategy,
+ const char *criteria);
+ </synopsis>
+ <para>
+ Create query objects using <function>ZOOM_query_create</function>
+ and destroy them by calling <function>ZOOM_query_destroy</function>.
+ RPN-queries can be specified in <link linkend="PQF">PQF</link>
+ notation by using the
+ function <function>ZOOM_query_prefix</function>.
+ The <function>ZOOM_query_cql</function> specifies a CQL
+ query to be sent to the server/target.
+ More query types will be added in future versions of &yaz;, such as
+ <link linkend="CCL">CCL</link> to RPN-mapping, native CCL query,
+ etc. In addition to a search, a sort criteria may be set. Function
+ <function>ZOOM_query_sortby</function> enables Z39.50 sorting and
+ it takes sort criteria using the same string notation as
+ yaz-client's <link linkend="sortspec">sort command</link>.
+ </para>
+ <para id="zoom.query.sortby2">
+ <function>ZOOM_query_sortby2</function> is similar to
+ <function>ZOOM_query_sortby</function> but allows a strategy for
+ sorting. The reason for the strategy parameter is that some
+ protocols offers multiple ways of performing sorting.
+ For example, Z39.50 has the standard sort, which is performed after
+ search on an existing result set.
+ It's also possible to use CQL in Z39.50 as the query type and use
+ CQL's SORTBY keyword. Finally, Index Data's
+ Zebra server also allows sorting to be specified as part of RPN (Type 7).
+ </para>
+ <table id="zoom-sort-strategy" frame="top">
+ <title>ZOOM sort strategy</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="name"/>
+ <colspec colwidth="5*" colname="description"/>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>z39.50</entry><entry>Z39.50 resultset sort</entry>
+ </row>
+ <row>
+ <entry>type7</entry><entry>Sorting embedded in RPN(Type-7)</entry>
+ </row>
+ <row>
+ <entry>cql</entry><entry>CQL SORTBY</entry>
+ </row>
+ <row>
+ <entry>sru11</entry><entry>SRU sortKeys parameter</entry>
+ </row>
+ <row>
+ <entry>solr</entry><entry>Solr sort</entry>
+ </row>
+ <row>
+ <entry>embed</entry><entry>type7 for Z39.50, cql for SRU,
+ solr for Solr protocol</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ <sect1 id="zoom.resultsets"><title>Result sets</title>
+ <para>
+ The result set object is a container for records returned from
+ a target.
+ </para>
+ <synopsis>
+ ZOOM_resultset ZOOM_connection_search(ZOOM_connection, ZOOM_query q);
+
+ ZOOM_resultset ZOOM_connection_search_pqf(ZOOM_connection c,
+ const char *q);
+ void ZOOM_resultset_destroy(ZOOM_resultset r);
+ </synopsis>
+ <para>
+ Function <function>ZOOM_connection_search</function> creates
+ a result set given a connection and query.
+ Destroy a result set by calling
+ <function>ZOOM_resultset_destroy</function>.
+ Simple clients may using PQF only may use function
+ <function>ZOOM_connection_search_pqf</function> in which case
+ creating query objects is not necessary.
+ </para>
+ <synopsis>
+ void ZOOM_resultset_option_set(ZOOM_resultset r,
+ const char *key, const char *val);
+
+ const char *ZOOM_resultset_option_get(ZOOM_resultset r, const char *key);
+
+ size_t ZOOM_resultset_size(ZOOM_resultset r);
+ </synopsis>
+ <para>
+ Functions <function>ZOOM_resultset_options_set</function> and
+ <function>ZOOM_resultset_get</function> sets and gets an option
+ for a result set similar to <function>ZOOM_connection_option_get</function>
+ and <function>ZOOM_connection_option_set</function>.
+ </para>
+ <para>
+ The number of hits also called result-count is returned by
+ function <function>ZOOM_resultset_size</function>.
+ </para>
+ <table id="zoom.resultset.options"
+ frame="top"><title>ZOOM Result set Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="2*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ start</entry><entry>Offset of first record to be
+ retrieved from target. First record has offset 0 unlike the
+ protocol specifications where first record has position 1.
+ This option affects ZOOM_resultset_search and
+ ZOOM_resultset_search_pqf and must be set before any of
+ these functions are invoked. If a range of
+ records must be fetched manually after search,
+ function ZOOM_resultset_records should be used.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ count</entry><entry>Number of records to be retrieved.
+ This option affects ZOOM_resultset_search and
+ ZOOM_resultset_search_pqf and must be set before any of
+ these functions are invoked.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ presentChunk</entry><entry>The number of records to be
+ requested from the server in each chunk (present request). The
+ value 0 means to request all the records in a single chunk.
+ (The old <literal>step</literal>
+ option is also supported for the benefit of old applications.)
+ </entry><entry>0</entry></row>
+ <row><entry>
+ elementSetName</entry><entry>Element-Set name of records.
+ Most targets should honor element set name <literal>B</literal>
+ and <literal>F</literal> for brief and full respectively.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ preferredRecordSyntax</entry><entry>Preferred Syntax, such as
+ <literal>USMARC</literal>, <literal>SUTRS</literal>, etc.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ schema</entry><entry>Schema for retrieval, such as
+ <literal>Gils-schema</literal>, <literal>Geo-schema</literal>, etc.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ setname</entry><entry>Name of Result Set (Result Set ID).
+ If this option isn't set, the ZOOM module will automatically
+ allocate a result set name.
+ </entry><entry>default</entry></row>
+ <row><entry>
+ rpnCharset</entry><entry>Character set for RPN terms.
+ If this is set, ZOOM C will assume that the ZOOM application is
+ running UTF-8. Terms in RPN queries are then converted to the
+ rpnCharset. If this is unset, ZOOM C will not assume any encoding
+ of RPN terms and no conversion is performed.
+ </entry><entry>none</entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ For servers that support Search Info report, the following
+ options may be read using <function>ZOOM_resultset_get</function>.
+ This detailed information is read after a successful search has
+ completed.
+ </para>
+ <para>
+ This information is a list of of items, where each item is
+ information about a term or subquery. All items in the list
+ are prefixed by
+ <literal>SearchResult.</literal><replaceable>no</replaceable>
+ where no presents the item number (0=first, 1=second).
+ Read <literal>searchresult.size</literal> to determine the
+ number of items.
+ </para>
+ <table id="zoom.search.info.report.options"
+ frame="top"><title>Search Info Report Options</title>
+ <tgroup cols="2">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>searchresult.size</entry>
+ <entry>
+ number of search result entries. This option is-nonexistant
+ if no entries are returned by the server.
+ </entry>
+ </row>
+ <row>
+ <entry>searchresult.<replaceable>no</replaceable>.id</entry>
+ <entry>sub query ID</entry>
+ </row>
+ <row>
+ <entry>searchresult.<replaceable>no</replaceable>.count</entry>
+ <entry>result count for item (number of hits)</entry>
+ </row>
+ <row>
+ <entry>searchresult.<replaceable>no</replaceable>.subquery.term</entry>
+ <entry>subquery term</entry>
+ </row>
+ <row>
+ <entry>
+ searchresult.<replaceable>no</replaceable>.interpretation.term
+ </entry>
+ <entry>interpretation term</entry>
+ </row>
+ <row>
+ <entry>
+ searchresult.<replaceable>no</replaceable>.recommendation.term
+ </entry>
+ <entry>recommendation term</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <sect2 id="zoom.z3950.resultset.sort">
+ <title>Z39.50 Result-set Sort</title>
+ <synopsis>
+ void ZOOM_resultset_sort(ZOOM_resultset r,
+ const char *sort_type, const char *sort_spec);
+
+ int ZOOM_resultset_sort1(ZOOM_resultset r,
+ const char *sort_type, const char *sort_spec);
+ </synopsis>
+ <para>
+ <function>ZOOM_resultset_sort</function> and
+ <function>ZOOM_resultset_sort1</function> both sort an existing
+ result-set. The sort_type parameter is not use. Set it to "yaz".
+ The sort_spec is same notation as ZOOM_query_sortby and identical
+ to that offered by yaz-client's
+ <link linkend="sortspec">sort command</link>.
+ </para>
+ <para>
+ These functions only work for Z39.50. Use the more generic utility
+ <link linkend="zoom.query.sortby2">
+ <function>ZOOM_query_sortby2</function></link>
+ for other protocols (and even Z39.50).
+ </para>
+ </sect2>
+ <sect2 id="zoom.z3950.resultset.behavior">
+ <title>Z39.50 Protocol behavior</title>
+ <para>
+ The creation of a result set involves at least a SearchRequest
+ - SearchResponse protocol handshake. Following that, if a sort
+ criteria was specified as part of the query, a SortRequest -
+ SortResponse handshake takes place. Note that it is necessary to
+ perform sorting before any retrieval takes place, so no records will
+ be returned from the target as part of the SearchResponse because these
+ would be unsorted. Hence, piggyback is disabled when sort criteria
+ are set. Following Search - and a possible sort - Retrieval takes
+ place - as one or more Present Requests/Response pairs being
+ transferred.
+ </para>
+ <para>
+ The API allows for two different modes for retrieval. A high level
+ mode which is somewhat more powerful and a low level one.
+ The low level is enabled when searching on a Connection object
+ for which the settings
+ <literal>smallSetUpperBound</literal>,
+ <literal>mediumSetPresentNumber</literal> and
+ <literal>largeSetLowerBound</literal> are set. The low level mode
+ thus allows you to precisely set how records are returned as part
+ of a search response as offered by the Z39.50 protocol.
+ Since the client may be retrieving records as part of the
+ search response, this mode doesn't work well if sorting is used.
+ </para>
+ <para>
+ The high-level mode allows you to fetch a range of records from
+ the result set with a given start offset. When you use this mode
+ the client will automatically use piggyback if that is possible
+ with the target and perform one or more present requests as needed.
+ Even if the target returns fewer records as part of a present response
+ because of a record size limit, etc. the client will repeat sending
+ present requests. As an example, if option <literal>start</literal>
+ is 0 (default) and <literal>count</literal> is 4, and
+ <literal>piggyback</literal> is 1 (default) and no sorting criteria
+ is specified, then the client will attempt to retrieve the 4
+ records as part the search response (using piggyback). On the other
+ hand, if either <literal>start</literal> is positive or if
+ a sorting criteria is set, or if <literal>piggyback</literal>
+ is 0, then the client will not perform piggyback but send Present
+ Requests instead.
+ </para>
+ <para>
+ If either of the options <literal>mediumSetElementSetName</literal> and
+ <literal>smallSetElementSetName</literal> are unset, the value
+ of option <literal>elementSetName</literal> is used for piggyback
+ searches. This means that for the high-level mode you only have
+ to specify one elementSetName option rather than three.
+ </para>
+ </sect2>
+ <sect2 id="zoom.sru.resultset.behavior">
+ <title>SRU Protocol behavior</title>
+ <para>
+ Current version of &yaz; does not take advantage of a result set id
+ returned by the SRU server. Future versions might do, however.
+ Since, the ZOOM driver does not save result set IDs any
+ present (retrieval) is transformed to a SRU SearchRetrieveRequest
+ with same query but, possibly, different offsets.
+ </para>
+ <para>
+ Option <literal>schema</literal> specifies SRU schema
+ for retrieval. However, options <literal>elementSetName</literal> and
+ <literal>preferredRecordSyntax</literal> are ignored.
+ </para>
+ <para>
+ Options <literal>start</literal> and <literal>count</literal>
+ are supported by SRU.
+ The remaining options
+ <literal>piggyback</literal>,
+ <literal>smallSetUpperBound</literal>,
+ <literal>largeSetLowerBound</literal>,
+ <literal>mediumSetPresentNumber</literal>,
+ <literal>mediumSetElementSetName</literal>,
+ <literal>smallSetElementSetName</literal> are
+ unsupported.
+ </para>
+ <para>
+ SRU supports CQL queries, <emphasis>not</emphasis> PQF.
+ If PQF is used, however, the PQF query is transferred anyway
+ using non-standard element <literal>pQuery</literal> in
+ SRU SearchRetrieveRequest.
+ </para>
+ <para>
+ Solr queries has to be done in Solr query format.
+ </para>
+ <para>
+ Unfortunately, SRU or Solr does not define a database setting. Hence,
+ <literal>databaseName</literal> is unsupported and ignored.
+ However, the path part in host parameter for functions
+ <function>ZOOM_connecton_new</function> and
+ <function>ZOOM_connection_connect</function> acts as a
+ database (at least for the &yaz; SRU server).
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.records">
+ <title>Records</title>
+ <para>
+ A record object is a retrieval record on the client side -
+ created from result sets.
+ </para>
+ <synopsis>
+ void ZOOM_resultset_records(ZOOM_resultset r,
+ ZOOM_record *recs,
+ size_t start, size_t count);
+ ZOOM_record ZOOM_resultset_record(ZOOM_resultset s, size_t pos);
+
+ const char *ZOOM_record_get(ZOOM_record rec, const char *type,
+ size_t *len);
+
+ int ZOOM_record_error(ZOOM_record rec, const char **msg,
+ const char **addinfo, const char **diagset);
+
+ ZOOM_record ZOOM_record_clone(ZOOM_record rec);
+
+ void ZOOM_record_destroy(ZOOM_record rec);
+ </synopsis>
+ <para>
+ References to temporary records are returned by functions
+ <function>ZOOM_resultset_records</function> or
+ <function>ZOOM_resultset_record</function>.
+ </para>
+ <para>
+ If a persistent reference to a record is desired
+ <function>ZOOM_record_clone</function> should be used.
+ It returns a record reference that should be destroyed
+ by a call to <function>ZOOM_record_destroy</function>.
+ </para>
+ <para>
+ A single record is returned by function
+ <function>ZOOM_resultset_record</function> that takes a
+ position as argument. First record has position zero.
+ If no record could be obtained <literal>NULL</literal> is returned.
+ </para>
+ <para>
+ Error information for a record can be checked with
+ <function>ZOOM_record_error</function> which returns non-zero
+ (error code) if record is in error, called <emphasis>Surrogate
+ Diagnostics</emphasis> in Z39.50.
+ </para>
+ <para>
+ Function <function>ZOOM_resultset_records</function> retrieves
+ a number of records from a result set. Parameter <literal>start</literal>
+ and <literal>count</literal> specifies the range of records to
+ be returned. Upon completion array
+ <literal>recs[0], ..recs[count-1]</literal>
+ holds record objects for the records. The array of records
+ <literal>recs</literal> should be allocated prior the call
+ <function>ZOOM_resultset_records</function>. Note that for those
+ records that couldn't be retrieved from the target
+ <literal>recs[ ..]</literal> is set to <literal>NULL</literal>.
+ </para>
+ <para id="zoom.record.get">
+ In order to extract information about a single record,
+ <function>ZOOM_record_get</function> is provided. The
+ function returns a pointer to certain record information. The
+ nature (type) of the pointer depends on the parameter,
+ <parameter>type</parameter>.
+ </para>
+ <para>
+ The <parameter>type</parameter> is a string of the format:
+ </para>
+ <para>
+ <replaceable>format</replaceable>[;charset=<replaceable>from</replaceable>[/<replaceable>opacfrom</replaceable>][,<replaceable>to</replaceable>]][;format=<replaceable>v</replaceable>]
+ </para>
+ <para>
+ where <replaceable>format</replaceable> specifies the format of the
+ returned record, <replaceable>from</replaceable>
+ specifies the character set of the record in its original form
+ (as returned by the server), <replaceable>to</replaceable> specifies
+ the output (returned)
+ character set encoding.
+ If <replaceable>to</replaceable> is omitted UTF-8 is assumed.
+ If charset is not given, then no character set conversion takes place.
+ </para>
+
+ <para>OPAC records may be returned in a different
+ set from the bibliographic MARC record. If this is this the case,
+ <replaceable>opacfrom</replaceable> should be set to the character set
+ of the OPAC record part.
+ </para>
+ <note>
+ <para>
+ Specifying the OPAC record character set requires YAZ 4.1.5 or later.
+ </para>
+ </note>
+ <para>
+ The format argument controls whether record data should be XML
+ pretty-printed (post process operation).
+ It is enabled only if format value <replaceable>v</replaceable> is
+ <literal>1</literal> and the record content is XML well-formed.
+ </para>
+ <para>
+ In addition, for certain types, the length
+ <literal>len</literal> passed will be set to the size in bytes of
+ the returned information.
+ </para>
+ <para>
+ The following are the supported values for <replaceable>form</replaceable>.
+ <variablelist>
+ <varlistentry><term><literal>database</literal></term>
+ <listitem><para>Database of record is returned
+ as a C null-terminated string. Return type
+ <literal>const char *</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>syntax</literal></term>
+ <listitem><para>The transfer syntax of the record is returned
+ as a C null-terminated string containing the symbolic name of
+ the record syntax, e.g. <literal>Usmarc</literal>. Return type
+ is
+ <literal>const char *</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>schema</literal></term>
+ <listitem><para>The schema of the record is returned
+ as a C null-terminated string. Return type is
+ <literal>const char *</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>render</literal></term>
+ <listitem><para>The record is returned in a display friendly
+ format. Upon completion buffer is returned
+ (type <literal>const char *</literal>) and length is stored in
+ <literal>*len</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>raw</literal></term>
+ <listitem><para>The record is returned in the internal
+ YAZ specific format. For GRS-1, Explain, and others, the
+ raw data is returned as type
+ <literal>Z_External *</literal> which is just the type for
+ the member <literal>retrievalRecord</literal> in
+ type <literal>NamePlusRecord</literal>.
+ For SUTRS and octet aligned record (including all MARCs) the
+ octet buffer is returned and the length of the buffer.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>xml</literal></term>
+ <listitem><para>The record is returned in XML if possible.
+ SRU, Solr and Z39.50 records with transfer syntax XML are
+ returned verbatim. MARC records are returned in
+ <ulink url="&url.marcxml;">
+ MARCXML
+ </ulink>
+ (converted from ISO2709 to MARCXML by YAZ).
+ OPAC records are also converted to XML and the
+ bibliographic record is converted to MARCXML (when possible).
+ GRS-1 records are not supported for this form.
+ Upon completion, the XML buffer is returned
+ (type <literal>const char *</literal>) and length is stored in
+ <literal>*len</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>opac</literal></term>
+ <listitem><para>OPAC information for record is returned in XML
+ if an OPAC record is present at the position given. If no
+ OPAC record is present, a NULL pointer is returned.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>txml</literal></term>
+ <listitem><para>The record is returned in TurboMARC if possible.
+ SRU and Z39.50 records with transfer syntax XML are
+ returned verbatim. MARC records are returned in
+ <link linkend="tools.turbomarc">
+ TurboMARC
+ </link>
+ (converted from ISO2709 to TurboMARC by YAZ).
+ Upon completion, the XML buffer is returned
+ (type <literal>const char *</literal>) and length is stored in
+ <literal>*len</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>json</literal></term>
+ <listitem><para>Like xml, but MARC records are converted to
+ <ulink url="&url.marc_in_json;">MARC-in-JSON</ulink>.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ <para>
+ Most
+ <ulink url="&url.marc21;">MARC21</ulink>
+ records uses the
+ <ulink url="&url.marc8;">MARC-8</ulink>
+ character set encoding.
+ An application that wishes to display in Latin-1 would use
+ <screen>
+ render; charset=marc8,iso-8859-1
+ </screen>
+ </para>
+ <sect2 id="zoom.z3950.record.behavior">
+ <title>Z39.50 Protocol behavior</title>
+ <para>
+ The functions <function>ZOOM_resultset_record</function> and
+ <function>ZOOM_resultset_records</function> inspects the client-side
+ record cache. Records not found in cache are fetched using
+ Present.
+ The functions may block (and perform network I/O) - even though option
+ <literal>async</literal> is 1, because they return records objects.
+ (and there's no way to return records objects without retrieving them!).
+ </para>
+ <para>
+ There is a trick, however, in the usage of function
+ <function>ZOOM_resultset_records</function> that allows for
+ delayed retrieval (and makes it non-blocking). By using
+ a null pointer for <parameter>recs</parameter> you're indicating
+ you're not interested in getting records objects
+ <emphasis>now</emphasis>.
+ </para>
+ </sect2>
+ <sect2 id="zoom.sru.record.behavior">
+ <title>SRU/Solr Protocol behavior</title>
+ <para>
+ The ZOOM driver for SRU/Solr treats records returned by a SRU/Solr server
+ as if they where Z39.50 records with transfer syntax XML and
+ no element set name or database name.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.facets"><title>Facets</title>
+ <para>
+ Facet operations is not part of the official ZOOM specification, but
+ is an Index Data extension for YAZ-based Z39.50 targets,
+ <ulink url="&url.solr;">Solr</ulink> and SRU 2.0 targets.
+
+ Facets may be requestd by the
+ <link linkend="zoom.facets.option">facets</link> option before a
+ search is sent.
+ For inspection of the returned facets, the following functions are
+ available:
+ </para>
+ <synopsis>
+ ZOOM_facet_field *ZOOM_resultset_facets(ZOOM_resultset r);
+
+ ZOOM_facet_field ZOOM_resultset_get_facet_field(ZOOM_resultset r,
+ const char *facet_name);
+
+ ZOOM_facet_field ZOOM_resultset_get_facet_field_by_index(ZOOM_resultset r,
+ int pos);
+
+ size_t ZOOM_resultset_facets_size(ZOOM_resultset r);
+
+ const char *ZOOM_facet_field_name(ZOOM_facet_field facet_field);
+
+ size_t ZOOM_facet_field_term_count(ZOOM_facet_field facet_field);
+
+ const char *ZOOM_facet_field_get_term(ZOOM_facet_field facet_field,
+ size_t idx, int *freq);
+ </synopsis>
+ <para>
+ References to temporary structures are returned by all functions.
+ They are only valid as long the Result set is valid.
+ <function>ZOOM_resultset_get_facet_field</function> or
+ <function>ZOOM_resultset_get_facet_field_by_index</function>.
+ <function>ZOOM_resultset_facets</function>.
+ <function>ZOOM_facet_field_name</function>.
+ <function>ZOOM_facet_field_get_term</function>.
+ </para>
+ <para id="zoom.resultset.get_facet_field">
+ A single Facet field is returned by function
+ <function>ZOOM_resultset_get_facet_field</function> or
+ <function>ZOOM_resultset_get_facet_field_by_index</function> that takes
+ a result set and facet name or positive index respectively. First
+ facet has position zero. If no facet could be obtained (invalid name
+ or index out of bounds) <literal>NULL</literal> is returned.
+ </para>
+ <para id="zoom.resultset.facets">
+ An array of facets field can be returned by
+ <function>ZOOM_resultset_facets</function>. The length of the array is
+ given by <function>ZOOM_resultset_facets_size</function>. The array is
+ zero-based and last entry will be at
+ <function>ZOOM_resultset_facets_size(result_set)</function>-1.
+ </para>
+ <para id="zoom.resultset.facets_names">
+ It is possible to interate over facets by name, by calling
+ <function>ZOOM_resultset_facets_names</function>.
+ This will return an const array of char * where each string can be used
+ as parameter for <function>ZOOM_resultset_get_facet_field</function>.
+ </para>
+ <para>
+ Function <function>ZOOM_facet_field_name</function> gets the request
+ facet name from a returned facet field.
+ </para>
+ <para>
+ Function <function>ZOOM_facet_field_get_term</function> returns the
+ idx'th term and term count for a facet field.
+ Idx must between 0 and
+ <function>ZOOM_facet_field_term_count</function>-1, otherwise the
+ returned reference will be <literal>NULL</literal>. On a valid idx, the
+ value of the freq reference will be the term count.
+ The <literal>freq</literal> parameter must be valid pointer to integer.
+ </para>
+ </sect1>
+ <sect1 id="zoom.scan"><title>Scan</title>
+ <para>
+ This section describes an interface for Scan. Scan is not an
+ official part of the ZOOM model yet. The result of a scan operation
+ is the <literal>ZOOM_scanset</literal> which is a set of terms
+ returned by a target.
+ </para>
+
+ <para>
+ The Scan interface is supported for both Z39.50, SRU and Solr.
+ </para>
+
+ <synopsis>
+ ZOOM_scanset ZOOM_connection_scan(ZOOM_connection c,
+ const char *startpqf);
+
+ ZOOM_scanset ZOOM_connection_scan1(ZOOM_connection c,
+ ZOOM_query q);
+
+ size_t ZOOM_scanset_size(ZOOM_scanset scan);
+
+ const char *ZOOM_scanset_term(ZOOM_scanset scan, size_t pos,
+ size_t *occ, size_t *len);
+
+ const char *ZOOM_scanset_display_term(ZOOM_scanset scan, size_t pos,
+ size_t *occ, size_t *len);
+
+ void ZOOM_scanset_destroy(ZOOM_scanset scan);
+
+ const char *ZOOM_scanset_option_get(ZOOM_scanset scan,
+ const char *key);
+
+ void ZOOM_scanset_option_set(ZOOM_scanset scan, const char *key,
+ const char *val);
+ </synopsis>
+ <para>
+ The scan set is created by function
+ <function>ZOOM_connection_scan</function> which performs a scan
+ operation on the connection using the specified
+ <parameter>startpqf</parameter>.
+ If the operation was successful, the size of the scan set can be
+ retrieved by a call to <function>ZOOM_scanset_size</function>.
+ Like result sets, the items are numbered 0,..size-1.
+ To obtain information about a particular scan term, call function
+ <function>ZOOM_scanset_term</function>. This function takes
+ a scan set offset <literal>pos</literal> and returns a pointer
+ to a <emphasis>raw term</emphasis> or <literal>NULL</literal> if
+ non-present.
+ If present, the <literal>occ</literal> and <literal>len</literal>
+ are set to the number of occurrences and the length
+ of the actual term respectively.
+ <function>ZOOM_scanset_display_term</function> is similar to
+ <function>ZOOM_scanset_term</function> except that it returns
+ the <emphasis>display term</emphasis> rather than the raw term.
+ In a few cases, the term is different from display term. Always
+ use the display term for display and the raw term for subsequent
+ scan operations (to get more terms, next scan result, etc).
+ </para>
+ <para>
+ A scan set may be freed by a call to function
+ <function>ZOOM_scanset_destroy</function>.
+ Functions <function>ZOOM_scanset_option_get</function> and
+ <function>ZOOM_scanset_option_set</function> retrieves and sets
+ an option respectively.
+ </para>
+ <para>
+ The <parameter>startpqf</parameter> is a subset of PQF, namely
+ the Attributes+Term part. Multiple <literal>@attr</literal> can
+ be used. For example to scan in title (complete) phrases:
+ <literallayout>
+ @attr 1=4 @attr 6=2 "science o"
+ </literallayout>
+ </para>
+ <para>
+ The <function>ZOOM_connecton_scan1</function> is a newer and
+ more generic alternative to <function>ZOOM_connection_scan</function>
+ which allows to use both CQL and PQF for Scan.
+ </para>
+ <table frame="top" id="zoom.scanset.options">
+ <title>ZOOM Scan Set Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="2*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ number</entry><entry>Number of Scan Terms requested in next scan.
+ After scan it holds the actual number of terms returned.
+ </entry><entry>20</entry></row>
+ <row><entry>
+ position</entry><entry>Preferred Position of term in response
+ in next scan; actual position after completion of scan.
+ </entry><entry>1</entry></row>
+ <row><entry>
+ stepSize</entry><entry>Step Size
+ </entry><entry>0</entry></row>
+ <row><entry>
+ scanStatus</entry><entry>An integer indicating the Scan Status
+ of last scan.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ rpnCharset</entry><entry>Character set for RPN terms.
+ If this is set, ZOOM C will assume that the ZOOM application is
+ running UTF-8. Terms in RPN queries are then converted to the
+ rpnCharset. If this is unset, ZOOM C will not assume any encoding
+ of RPN terms and no conversion is performed.
+ </entry><entry>none</entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ <sect1 id="zoom.extendedservices">
+ <title>Extended Services</title>
+ <para>
+ ZOOM offers an interface to a subset of the Z39.50 extended services
+ as well as a few privately defined ones:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Z39.50 Item Order (ILL).
+ See <xref linkend="zoom.item.order"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Record Update. This allows a client to insert, modify or delete
+ records.
+ See <xref linkend="zoom.record.update"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Database Create. This a non-standard feature. Allows a client
+ to create a database.
+ See <xref linkend="zoom.database.create"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Database Drop. This a non-standard feature. Allows a client
+ to delete/drop a database.
+ See <xref linkend="zoom.database.drop"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Commit operation. This a non-standard feature. Allows a client
+ to commit operations.
+ See <xref linkend="zoom.commit"/>.
+ </para>
+ </listitem>
+ <!-- all the ILL PDU options should go here too -->
+ </itemizedlist>
+ <para>
+ To create an extended service operation a <literal>ZOOM_package</literal>
+ must be created. The operation is a five step operation. The
+ package is created, package is configured by means of options,
+ the package is send, result is inspected (by means of options),
+ the package is destroyed.
+ </para>
+ <synopsis>
+ ZOOM_package ZOOM_connection_package(ZOOM_connection c,
+ ZOOM_options options);
+
+ const char *ZOOM_package_option_get(ZOOM_package p,
+ const char *key);
+ void ZOOM_package_option_set(ZOOM_package p, const char *key,
+ const char *val);
+ void ZOOM_package_send(ZOOM_package p, const char *type);
+
+ void ZOOM_package_destroy(ZOOM_package p);
+ </synopsis>
+ <para>
+ The <function>ZOOM_connection_package</function> creates a
+ package for the connection given using the options specified.
+ </para>
+ <para>
+ Functions <function>ZOOM_package_option_get</function> and
+ <function>ZOOM_package_option_set</function> gets and sets
+ options.
+ </para>
+ <para>
+ <function>ZOOM_package_send</function> sends
+ the package the via connection specified in
+ <function>ZOOM_connection_package</function>.
+ The <parameter>type</parameter> specifies the actual extended service
+ package type to be sent.
+ </para>
+ <table frame="top" id="zoom.extendedservices.options">
+ <title>Extended Service Common Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>package-name</entry>
+ <entry>Extended Service Request package name. Must be specified
+ as part of a request</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>user-id</entry>
+ <entry>User ID of Extended Service Package. Is a request option</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>function</entry>
+ <entry>
+ Function of package - one of <literal>create</literal>,
+ <literal>delete</literal>, <literal>modify</literal>. Is
+ a request option.
+ </entry>
+ <entry><literal>create</literal></entry>
+ </row>
+ <row>
+ <entry>waitAction</entry>
+ <entry>
+ Wait action for package. Possible values:
+ <literal>wait</literal>, <literal>waitIfPossible</literal>,
+ <literal>dontWait</literal> or <literal>dontReturnPackage</literal>.
+ </entry>
+ <entry><literal>waitIfPossible</literal></entry>
+ </row>
+ <row>
+ <entry>targetReference</entry>
+ <entry>
+ Target Reference. This is part of the response as returned
+ by the server. Read it after a successful operation.
+ </entry>
+ <entry><literal>none</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <sect2 id="zoom.item.order">
+ <title>Item Order</title>
+ <para>
+ For Item Order, type must be set to <literal>itemorder</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+
+ <table frame="top" id="zoom.item.order.options">
+ <title>Item Order Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>contact-name</entry>
+ <entry>ILL contact name</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>contact-phone</entry>
+ <entry>ILL contact phone</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>contact-email</entry>
+ <entry>ILL contact email</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>itemorder-item</entry>
+ <entry>Position for item (record) requested. An integer</entry>
+ <entry>1</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ <sect2 id="zoom.record.update">
+ <title>Record Update</title>
+ <para>
+ For Record Update, type must be set to <literal>update</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+ <table frame="top" id="zoom.record.update.options">
+ <title>Record Update Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>action</entry>
+ <entry>
+ The update action. One of
+ <literal>specialUpdate</literal>,
+ <literal>recordInsert</literal>,
+ <literal>recordReplace</literal>,
+ <literal>recordDelete</literal>,
+ <literal>elementUpdate</literal>.
+ </entry>
+ <entry><literal>specialUpdate (recordInsert for updateVersion=1 which does not support specialUpdate)</literal></entry>
+ </row>
+ <row>
+ <entry>recordIdOpaque</entry>
+ <entry>Opaque Record ID</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>recordIdNumber</entry>
+ <entry>Record ID number</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>record</entry>
+ <entry>The record itself</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>recordOpaque</entry>
+ <entry>Specifies an opaque record which is
+ encoded as an ASN.1 ANY type with the OID as tiven by option
+ <literal>syntax</literal> (see below).
+ Option <literal>recordOpaque</literal> is an alternative
+ to record - and <literal>record</literal> option (above) is
+ ignored if recordOpaque is set. This option is only available in
+ YAZ 3.0.35 and later and is meant to facilitate Updates with
+ servers from OCLC.
+ </entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>syntax</entry>
+ <entry>The record syntax (transfer syntax). Is a string that
+ is a known record syntax.
+ </entry>
+ <entry>no syntax</entry>
+ </row>
+ <row>
+ <entry>databaseName</entry>
+ <entry>Database from connection object</entry>
+ <entry>Default</entry>
+ </row>
+ <row>
+ <entry>correlationInfo.note</entry>
+ <entry>Correlation Info Note (string)</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>correlationInfo.id</entry>
+ <entry>Correlation Info ID (integer)</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>elementSetName</entry>
+ <entry>Element Set for Record</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>updateVersion</entry>
+ <entry>Record Update version which holds one of the values
+ 1, 2 or 3. Each version has a distinct OID:
+ 1.2.840.10003.9.5
+ (<ulink url="&url.z39.50.extupdate1;">first version</ulink>) ,
+ 1.2.840.10003.9.5.1
+ (second version) and
+ 1.2.840.10003.9.5.1.1
+ (<ulink url="&url.z39.50.extupdate3;">third and
+ newest version</ulink>).
+ </entry>
+ <entry>3</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ </sect2>
+
+ <sect2 id="zoom.database.create"><title>Database Create</title>
+ <para>
+ For Database Create, type must be set to <literal>create</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+
+ <table frame="top" id="zoom.database.create.options">
+ <title>Database Create Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>databaseName</entry>
+ <entry>Database from connection object</entry>
+ <entry>Default</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ <sect2 id="zoom.database.drop">
+ <title>Database Drop</title>
+ <para>
+ For Database Drop, type must be set to <literal>drop</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+ <table frame="top" id="zoom.database.drop.options">
+ <title>Database Drop Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>databaseName</entry>
+ <entry>Database from connection object</entry>
+ <entry>Default</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ <sect2 id="zoom.commit">
+ <title>Commit Operation</title>
+ <para>
+ For Commit, type must be set to <literal>commit</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+ </sect2>
+ <sect2 id="zoom.extended.services.behavior">
+ <title>Protocol behavior</title>
+ <para>
+ All the extended services are Z39.50-only.
+ </para>
+ <note>
+ <para>
+ The database create, drop and commit services are privately defined
+ operations.
+ Refer to <filename>esadmin.asn</filename> in YAZ for the ASN.1
+ definitions.
+ </para>
+ </note>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.options">
+ <title>Options</title>
+ <para>
+ Most &zoom; objects provide a way to specify options to change behavior.
+ From an implementation point of view a set of options is just like
+ an associative array / hash.
+ </para>
+ <synopsis>
+ ZOOM_options ZOOM_options_create(void);
+
+ ZOOM_options ZOOM_options_create_with_parent(ZOOM_options parent);
+
+ void ZOOM_options_destroy(ZOOM_options opt);
+ </synopsis>
+ <synopsis>
+ const char *ZOOM_options_get(ZOOM_options opt, const char *name);
+
+ void ZOOM_options_set(ZOOM_options opt, const char *name,
+ const char *v);
+ </synopsis>
+ <synopsis>
+ typedef const char *(*ZOOM_options_callback)
+ (void *handle, const char *name);
+
+ ZOOM_options_callback
+ ZOOM_options_set_callback(ZOOM_options opt,
+ ZOOM_options_callback c,
+ void *handle);
+ </synopsis>
+ </sect1>
+ <sect1 id="zoom.queryconversions">
+ <title>Query conversions</title>
+ <synopsis>
+ int ZOOM_query_cql2rpn(ZOOM_query s, const char *cql_str,
+ ZOOM_connection conn);
+
+ int ZOOM_query_ccl2rpn(ZOOM_query s, const char *ccl_str,
+ const char *config,
+ int *ccl_error, const char **error_string,
+ int *error_pos);
+ </synopsis>
+ <para>
+ <function>ZOOM_query_cql2rpn</function> translates the CQL string,
+ client-side, into RPN which may be passed to the server.
+ This is useful for server's that don't themselves
+ support CQL, for which <function>ZOOM_query_cql</function> is useless.
+ `conn' is used only as a place to stash diagnostics if compilation
+ fails; if this information is not needed, a null pointer may be used.
+ The CQL conversion is driven by option <literal>cqlfile</literal> from
+ connection conn. This specifies a conversion file (eg pqf.properties)
+ which <emphasis>must</emphasis> be present.
+ </para>
+ <para>
+ <function>ZOOM_query_ccl2rpn</function> translates the CCL string,
+ client-side, into RPN which may be passed to the server.
+ The conversion is driven by the specification given by
+ <literal>config</literal>. Upon completion 0 is returned on success; -1
+ is returned on on failure. Om failure <literal>error_string</literal> and
+ <literal>error_pos</literal> holds error message and position of
+ first error in original CCL string.
+ </para>
+ </sect1>
+ <sect1 id="zoom.events"><title>Events</title>
+ <para>
+ If you're developing non-blocking applications, you have to deal
+ with events.
+ </para>
+ <synopsis>
+ int ZOOM_event(int no, ZOOM_connection *cs);
+ </synopsis>
+ <para>
+ The <function>ZOOM_event</function> executes pending events for
+ a number of connections. Supply the number of connections in
+ <literal>no</literal> and an array of connections in
+ <literal>cs</literal> (<literal>cs[0] ... cs[no-1]</literal>).
+ A pending event could be a sending a search, receiving a response,
+ etc.
+ When an event has occurred for one of the connections, this function
+ returns a positive integer <literal>n</literal> denoting that an event
+ occurred for connection <literal>cs[n-1]</literal>.
+ When no events are pending for the connections, a value of zero is
+ returned.
+ To ensure that all outstanding requests are performed call this function
+ repeatedly until zero is returned.
+ </para>
+ <para>
+ If <function>ZOOM_event</function> returns and returns non-zero, the
+ last event that occurred can be expected.
+ </para>
+ <synopsis>
+ int ZOOM_connection_last_event(ZOOM_connection cs);
+ </synopsis>
+ <para>
+ <function>ZOOM_connection_last_event</function> returns an event type
+ (integer) for the last event.
+ </para>
+
+ <table frame="top" id="zoom.event.ids">
+ <title>ZOOM Event IDs</title>
+ <tgroup cols="2">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Event</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>ZOOM_EVENT_NONE</entry>
+ <entry>No event has occurred</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_CONNECT</entry>
+ <entry>TCP/IP connect has initiated</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_SEND_DATA</entry>
+ <entry>Data has been transmitted (sending)</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_DATA</entry>
+ <entry>Data has been received)</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_TIMEOUT</entry>
+ <entry>Timeout</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_UNKNOWN</entry>
+ <entry>Unknown event</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_SEND_APDU</entry>
+ <entry>An APDU has been transmitted (sending)</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_APDU</entry>
+ <entry>An APDU has been received</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_RECORD</entry>
+ <entry>A result-set record has been received</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_SEARCH</entry>
+ <entry>A search result been received</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ </chapter>
+ <chapter id="server">
+ <title>Generic server</title>
+ <sect1 id="server.introduction"><title>Introduction</title>
+ <para>
+ If you aren't into documentation, a good way to learn how the
+ back end interface works is to look at the <filename>backend.h</filename>
+ file. Then, look at the small dummy-server in
+ <filename>ztest/ztest.c</filename>. The <filename>backend.h</filename>
+ file also makes a good reference, once you've chewed your way through
+ the prose of this file.
+ </para>
+ <para>
+ If you have a database system that you would like to make available by
+ means of Z39.50 or SRU, &yaz; basically offers your two options. You
+ can use the APIs provided by the &asn;, &odr;, and &comstack;
+ modules to
+ create and decode PDUs, and exchange them with a client.
+ Using this low-level interface gives you access to all fields and
+ options of the protocol, and you can construct your server as close
+ to your existing database as you like.
+ It is also a fairly involved process, requiring
+ you to set up an event-handling mechanism, protocol state machine,
+ etc. To simplify server implementation, we have implemented a compact
+ and simple, but reasonably full-functioned server-frontend that will
+ handle most of the protocol mechanics, while leaving you to
+ concentrate on your database interface.
+ </para>
+ <note>
+ <para>
+ The backend interface was designed in anticipation of a specific
+ integration task, while still attempting to achieve some degree of
+ generality. We realize fully that there are points where the
+ interface can be improved significantly. If you have specific
+ functions or parameters that you think could be useful, send us a
+ mail (or better, sign on to the mailing list referred to in the
+ top-level README file). We will try to fit good suggestions into future
+ releases, to the extent that it can be done without requiring
+ too many structural changes in existing applications.
+ </para>
+ </note>
+ <note>
+ <para>
+ The &yaz; server does not support XCQL.
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="server.frontend">
+ <title>The Database Frontend</title>
+ <para>
+ We refer to this software as a generic database frontend. Your
+ database system is the <emphasis>backend database</emphasis>, and the
+ interface between the two is called the <emphasis>backend API</emphasis>.
+ The backend API consists of a small number of function handlers and
+ structure definitions. You are required to provide the
+ <function>main()</function> routine for the server (which can be
+ quite simple), as well as a set of handlers to match each of the
+ prototypes.
+ The interface functions that you write can use any mechanism you like
+ to communicate with your database system: You might link the whole
+ thing together with your database application and access it by
+ function calls; you might use IPC to talk to a database server
+ somewhere; or you might link with third-party software that handles
+ the communication for you (like a commercial database client library).
+ At any rate, the handlers will perform the tasks of:
+ </para>
+ <itemizedlist>
+ <listitem><para>
+ Initialization.
+ </para></listitem>
+ <listitem><para>
+ Searching.
+ </para></listitem>
+ <listitem><para>
+ Fetching records.
+ </para></listitem>
+ <listitem><para>
+ Scanning the database index (optional - if you wish to implement SCAN).
+ </para></listitem>
+ <listitem><para>
+ Extended Services (optional).
+ </para></listitem>
+ <listitem><para>
+ Result-Set Delete (optional).
+ </para></listitem>
+ <listitem><para>
+ Result-Set Sort (optional).
+ </para></listitem>
+ <listitem><para>
+ Return Explain for SRU (optional).
+ </para></listitem>
+ </itemizedlist>
+ <para>
+ (more functions will be added in time to support as much of
+ Z39.50-1995 as possible).
+ </para>
+ </sect1>
+ <sect1 id="server.backend">
+ <title>The Backend API</title>
+ <para>
+ The header file that you need to use the interface are in the
+ <filename>include/yaz</filename> directory. It's called
+ <filename>backend.h</filename>. It will include other files from
+ the <filename>include/yaz</filename> directory, so you'll
+ probably want to use the -I option of your compiler to tell it
+ where to find the files. When you run
+ <literal>make</literal> in the top-level &yaz; directory,
+ everything you need to create your server is to link with the
+ <filename>lib/libyaz.la</filename> library.
+ </para>
+ </sect1>
+ <sect1 id="server.main">
+ <title>Your main() Routine</title>
+ <para>
+ As mentioned, your <function>main()</function> routine can be quite brief.
+ If you want to initialize global parameters, or read global configuration
+ tables, this is the place to do it. At the end of the routine, you should
+ call the function
+ </para>
+ <synopsis>
+int statserv_main(int argc, char **argv,
+ bend_initresult *(*bend_init)(bend_initrequest *r),
+ void (*bend_close)(void *handle));
+ </synopsis>
+ <para>
+ The third and fourth arguments are pointers to handlers. Handler
+ <function>bend_init</function> is called whenever the server receives
+ an Initialize Request, so it serves as a Z39.50 session initializer. The
+ <function>bend_close</function> handler is called when the session is
+ closed.
+ </para>
+ <para>
+ <function>statserv_main</function> will establish listening sockets
+ according to the parameters given. When connection requests are received,
+ the event handler will typically <function>fork()</function> and
+ create a sub-process to handle a new connection.
+ Alternatively the server may be setup to create threads for each
+ connection.
+ If you do use global variables and forking, you should be aware, then,
+ that these cannot be shared between associations, unless you explicitly
+ disable forking by command line parameters.
+ </para>
+ <para>
+ The server provides a mechanism for controlling some of its behavior
+ without using command-line options. The function
+ </para>
+ <synopsis>
+ statserv_options_block *statserv_getcontrol(void);
+ </synopsis>
+ <para>
+ will return a pointer to a <literal>struct statserv_options_block</literal>
+ describing the current default settings of the server. The structure
+ contains these elements:
+ <variablelist>
+ <varlistentry>
+ <term><literal>int dynamic</literal></term>
+ <listitem><para>
+ A boolean value, which determines whether the server
+ will fork on each incoming request (TRUE), or not (FALSE). Default is
+ TRUE. This flag is only read by UNIX-based servers (WIN32 based servers
+ doesn't fork).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int threads</literal></term>
+ <listitem><para>
+ A boolean value, which determines whether the server
+ will create a thread on each incoming request (TRUE), or not (FALSE).
+ Default is FALSE. This flag is only read by UNIX-based servers
+ that offer POSIX Threads support.
+ WIN32-based servers always operate in threaded mode.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int inetd</literal></term>
+ <listitem><para>
+ A boolean value, which determines whether the server
+ will operates under a UNIX INET daemon (inetd). Default is FALSE.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char logfile[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>File for diagnostic output ("": stderr).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char apdufile[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>
+ Name of file for logging incoming and outgoing APDUs
+ ("": don't log APDUs, "-":
+ <literal>stderr</literal>).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char default_listen[1024]</literal></term>
+ <listitem><para>Same form as the command-line specification of
+ listener address. "": no default listener address.
+ Default is to listen at "tcp:@:9999". You can only
+ specify one default listener address in this fashion.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>enum oid_proto default_proto;</literal></term>
+ <listitem><para>Either <literal>PROTO_Z3950</literal> or
+ <literal>PROTO_SR</literal>.
+ Default is <literal>PROTO_Z39_50</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int idle_timeout;</literal></term>
+ <listitem><para>Maximum session idle-time, in minutes. Zero indicates
+ no (infinite) timeout. Default is 15 minutes.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int maxrecordsize;</literal></term>
+ <listitem><para>Maximum permissible record (message) size. Default
+ is 64 MB. This amount of memory will only be allocated if a
+ client requests a very large amount of records in one operation
+ (or a big record).
+ Set it to a lower number if you are worried about resource
+ consumption on your host system.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char configname[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>Passed to the backend when a new connection is received.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char setuid[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>Set user id to the user specified, after binding
+ the listener addresses.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>void (*bend_start)(struct statserv_options_block *p)</literal>
+ </term>
+ <listitem><para>Pointer to function which is called after the
+ command line options have been parsed - but before the server
+ starts listening.
+ For forked UNIX servers this handler is called in the mother
+ process; for threaded servers this handler is called in the
+ main thread.
+ The default value of this pointer is NULL in which case it
+ isn't invoked by the frontend server.
+ When the server operates as an NT service this handler is called
+ whenever the service is started.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>void (*bend_stop)(struct statserv_options_block *p)</literal>
+ </term>
+ <listitem><para>Pointer to function which is called whenever the server
+ has stopped listening for incoming connections. This function pointer
+ has a default value of NULL in which case it isn't called.
+ When the server operates as an NT service this handler is called
+ whenever the service is stopped.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>void *handle</literal></term>
+ <listitem><para>User defined pointer (default value NULL).
+ This is a per-server handle that can be used to specify "user-data".
+ Do not confuse this with the session-handle as returned by bend_init.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ The pointer returned by <literal>statserv_getcontrol</literal> points to
+ a static area. You are allowed to change the contents of the structure,
+ but the changes will not take effect before you call
+ </para>
+ <synopsis>
+void statserv_setcontrol(statserv_options_block *block);
+ </synopsis>
+ <note>
+ <para>
+ that you should generally update this structure before calling
+ <function>statserv_main()</function>.
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="server.backendfunctions">
+ <title>The Backend Functions</title>
+ <para>
+ For each service of the protocol, the backend interface declares one or
+ two functions. You are required to provide implementations of the
+ functions representing the services that you wish to implement.
+ </para>
+ <sect2 id="server.init">
+ <title>Init</title>
+ <synopsis>
+bend_initresult (*bend_init)(bend_initrequest *r);
+ </synopsis>
+ <para>
+ This handler is called once for each new connection request, after
+ a new process/thread has been created, and an Initialize Request has
+ been received from the client. The pointer to the
+ <function>bend_init</function> handler is passed in the call to
+ <function>statserv_start</function>.
+ </para>
+ <para>
+ This handler is also called when operating in SRU mode - when
+ a connection has been made (even though SRU does not offer
+ this service).
+ </para>
+ <para>
+ Unlike previous versions of YAZ, the <function>bend_init</function> also
+ serves as a handler that defines the Z39.50 services that the backend
+ wish to support. Pointers to <emphasis>all</emphasis> service handlers,
+ including search - and fetch must be specified here in this handler.
+ </para>
+ <para>
+ The request - and result structures are defined as
+ </para>
+ <synopsis>
+typedef struct bend_initrequest
+{
+ /** \brief user/name/password to be read */
+ Z_IdAuthentication *auth;
+ /** \brief encoding stream (for results) */
+ ODR stream;
+ /** \brief printing stream */
+ ODR print;
+ /** \brief decoding stream (use stream for results) */
+ ODR decode;
+ /** \brief reference ID */
+ Z_ReferenceId *referenceId;
+ /** \brief peer address of client */
+ char *peer_name;
+
+ /** \brief character set and language negotiation
+
+ see include/yaz/z-charneg.h
+ */
+ Z_CharSetandLanguageNegotiation *charneg_request;
+
+ /** \brief character negotiation response */
+ Z_External *charneg_response;
+
+ /** \brief character set (encoding) for query terms
+
+ This is NULL by default. It should be set to the native character
+ set that the backend assumes for query terms */
+ char *query_charset;
+
+ /** \brief whehter query_charset also applies to recors
+
+ Is 0 (No) by default. Set to 1 (yes) if records is in the same
+ character set as queries. If in doubt, use 0 (No).
+ */
+ int records_in_same_charset;
+
+ char *implementation_id;
+ char *implementation_name;
+ char *implementation_version;
+
+ /** \brief Z39.50 sort handler */
+ int (*bend_sort)(void *handle, bend_sort_rr *rr);
+ /** \brief SRU/Z39.50 search handler */
+ int (*bend_search)(void *handle, bend_search_rr *rr);
+ /** \brief SRU/Z39.50 fetch handler */
+ int (*bend_fetch)(void *handle, bend_fetch_rr *rr);
+ /** \brief SRU/Z39.50 present handler */
+ int (*bend_present)(void *handle, bend_present_rr *rr);
+ /** \brief Z39.50 extended services handler */
+ int (*bend_esrequest) (void *handle, bend_esrequest_rr *rr);
+ /** \brief Z39.50 delete result set handler */
+ int (*bend_delete)(void *handle, bend_delete_rr *rr);
+ /** \brief Z39.50 scan handler */
+ int (*bend_scan)(void *handle, bend_scan_rr *rr);
+ /** \brief Z39.50 segment facility handler */
+ int (*bend_segment)(void *handle, bend_segment_rr *rr);
+ /** \brief SRU explain handler */
+ int (*bend_explain)(void *handle, bend_explain_rr *rr);
+ /** \brief SRU scan handler */
+ int (*bend_srw_scan)(void *handle, bend_scan_rr *rr);
+ /** \brief SRU record update handler */
+ int (*bend_srw_update)(void *handle, bend_update_rr *rr);
+
+ /** \brief whether named result sets are supported (0=disable, 1=enable) */
+ int named_result_sets;
+} bend_initrequest;
+
+typedef struct bend_initresult
+{
+ int errcode; /* 0==OK */
+ char *errstring; /* system error string or NULL */
+ void *handle; /* private handle to the backend module */
+} bend_initresult;
+ </synopsis>
+ <para>
+ In general, the server frontend expects that the
+ <literal>bend_*result</literal> pointer that you return is valid at
+ least until the next call to a <literal>bend_* function</literal>.
+ This applies to all of the functions described herein. The parameter
+ structure passed to you in the call belongs to the server frontend, and
+ you should not make assumptions about its contents after the current
+ function call has completed. In other words, if you want to retain any
+ of the contents of a request structure, you should copy them.
+ </para>
+ <para>
+ The <literal>errcode</literal> should be zero if the initialization of
+ the backend went well. Any other value will be interpreted as an error.
+ The <literal>errstring</literal> isn't used in the current version, but
+ one option would be to stick it in the initResponse as a VisibleString.
+ The <literal>handle</literal> is the most important parameter. It should
+ be set to some value that uniquely identifies the current session to
+ the backend implementation. It is used by the frontend server in any
+ future calls to a backend function.
+ The typical use is to set it to point to a dynamically allocated state
+ structure that is private to your backend module.
+ </para>
+ <para>
+ The <literal>auth</literal> member holds the authentication information
+ part of the Z39.50 Initialize Request. Interpret this if your serves
+ requires authentication.
+ </para>
+ <para>
+ The members <literal>peer_name</literal>,
+ <literal>implementation_id</literal>,
+ <literal>implementation_name</literal> and
+ <literal>implementation_version</literal> holds
+ DNS of client, ID of implementor, name
+ of client (Z39.50) implementation - and version.
+ </para>
+ <para>
+ The <literal>bend_</literal> - members are set to NULL when
+ <function>bend_init</function> is called. Modify the pointers by
+ setting them to point to backend functions.
+ </para>
+ </sect2>
+ <sect2 id="server.search.retrieve">
+ <title>Search and Retrieve</title>
+ <para>
+ We now describe the handlers that are required to support search -
+ and retrieve. You must support two functions - one for search - and one
+ for fetch (retrieval of one record). If desirable you can provide a
+ third handler which is called when a present request is received which
+ allows you to optimize retrieval of multiple-records.
+ </para>
+ <synopsis>
+int (*bend_search) (void *handle, bend_search_rr *rr);
+
+typedef struct {
+ char *setname; /* name to give to this set */
+ int replace_set; /* replace set, if it already exists */
+ int num_bases; /* number of databases in list */
+ char **basenames; /* databases to search */
+ Z_ReferenceId *referenceId;/* reference ID */
+ Z_Query *query; /* query structure */
+ ODR stream; /* encode stream */
+ ODR decode; /* decode stream */
+ ODR print; /* print stream */
+
+ bend_request request;
+ bend_association association;
+ int *fd;
+ int hits; /* number of hits */
+ int errcode; /* 0==OK */
+ char *errstring; /* system error string or NULL */
+ Z_OtherInformation *search_info; /* additional search info */
+ char *srw_sortKeys; /* holds SRU/SRW sortKeys info */
+ char *srw_setname; /* holds SRU/SRW generated resultsetID */
+ int *srw_setnameIdleTime; /* holds SRU/SRW life-time */
+ int estimated_hit_count; /* if hit count is estimated */
+ int partial_resultset; /* if result set is partial */
+} bend_search_rr;
+ </synopsis>
+ <para>
+ The <function>bend_search</function> handler is a fairly close
+ approximation of a protocol Z39.50 Search Request - and Response PDUs
+ The <literal>setname</literal> is the resultSetName from the protocol.
+ You are required to establish a mapping between the set name and whatever
+ your backend database likes to use.
+ Similarly, the <literal>replace_set</literal> is a boolean value
+ corresponding to the resultSetIndicator field in the protocol.
+ <literal>num_bases/basenames</literal> is a length of/array of character
+ pointers to the database names provided by the client.
+ The <literal>query</literal> is the full query structure as defined in
+ the protocol ASN.1 specification.
+ It can be either of the possible query types, and it's up to you to
+ determine if you can handle the provided query type.
+ Rather than reproduce the C interface here, we'll refer you to the
+ structure definitions in the file
+ <filename>include/yaz/z-core.h</filename>. If you want to look at the
+ attributeSetId OID of the RPN query, you can either match it against
+ your own internal tables, or you can use the <link linkend="tools.oid">
+ OID tools</link>.
+ </para>
+ <para>
+ The structure contains a number of hits, and an
+ <literal>errcode/errstring</literal> pair. If an error occurs
+ during the search, or if you're unhappy with the request, you should
+ set the errcode to a value from the BIB-1 diagnostic set. The value
+ will then be returned to the user in a nonsurrogate diagnostic record
+ in the response. The <literal>errstring</literal>, if provided, will
+ go in the addinfo field. Look at the protocol definition for the
+ defined error codes, and the suggested uses of the addinfo field.
+ </para>
+ <para>
+ The <function>bend_search</function> handler is also called when
+ the frontend server receives a SRU SearchRetrieveRequest.
+ For SRU, a CQL query is usually provided by the client.
+ The CQL query is available as part of <literal>Z_Query</literal>
+ structure (note that CQL is now part of Z39.50 via an external).
+ To support CQL in existing implementations that only do Type-1,
+ we refer to the CQL-to-PQF tool described
+ <link linkend="cql.to.pqf">here</link>.
+ </para>
+ <para>
+ To maintain backwards compatibility, the frontend server
+ of yaz always assume that error codes are BIB-1 diagnostics.
+ For SRU operation, a Bib-1 diagnostic code is mapped to
+ SRU diagnostic.
+ </para>
+ <synopsis>
+int (*bend_fetch) (void *handle, bend_fetch_rr *rr);
+
+typedef struct bend_fetch_rr {
+ char *setname; /* set name */
+ int number; /* record number */
+ Z_ReferenceId *referenceId;/* reference ID */
+ Odr_oid *request_format; /* format, transfer syntax (OID) */
+ Z_RecordComposition *comp; /* Formatting instructions */
+ ODR stream; /* encoding stream - memory source if req */
+ ODR print; /* printing stream */
+
+ char *basename; /* name of database that provided record */
+ int len; /* length of record or -1 if structured */
+ char *record; /* record */
+ int last_in_set; /* is it? */
+ Odr_oid *output_format; /* response format/syntax (OID) */
+ int errcode; /* 0==success */
+ char *errstring; /* system error string or NULL */
+ int surrogate_flag; /* surrogate diagnostic */
+ char *schema; /* string record schema input/output */
+} bend_fetch_rr;
+ </synopsis>
+ <para>
+ The frontend server calls the <function>bend_fetch</function> handler
+ when it needs database records to fulfill a Z39.50 Search Request, a
+ Z39.50 Present Request or a SRU SearchRetrieveRequest.
+ The <literal>setname</literal> is simply the name of the result set
+ that holds the reference to the desired record.
+ The <literal>number</literal> is the offset into the set (with 1
+ being the first record in the set). The <literal>format</literal> field
+ is the record format requested by the client (See
+ <xref linkend="tools.oid"/>).
+ A value of NULL for <literal>format</literal> indicates that the
+ client did not request a specific format.
+ The <literal>stream</literal> argument is an &odr; stream which
+ should be used for allocating space for structured data records.
+ The stream will be reset when all records have been assembled, and
+ the response package has been transmitted.
+ For unstructured data, the backend is responsible for maintaining a
+ static or dynamic buffer for the record between calls.
+ </para>
+ <para>
+ If a SRU SearchRetrieveRequest is received by the frontend server,
+ the <literal>referenceId</literal> is NULL and the
+ <literal>format</literal> (transfer syntax) is the OID for XML.
+ The schema for SRU is stored in both the
+ <literal>Z_RecordComposition</literal>
+ structure and <literal>schema</literal> (simple string).
+ </para>
+ <para>
+ In the structure, the <literal>basename</literal> is the name of the
+ database that holds the
+ record. <literal>len</literal> is the length of the record returned, in
+ bytes, and <literal>record</literal> is a pointer to the record.
+ <literal>last_in_set</literal> should be nonzero only if the record
+ returned is the last one in the given result set.
+ <literal>errcode</literal> and <literal>errstring</literal>, if
+ given, will be interpreted as a global error pertaining to the
+ set, and will be returned in a non-surrogate-diagnostic.
+ If you wish to return the error as a surrogate-diagnostic
+ (local error) you can do this by setting
+ <literal>surrogate_flag</literal> to 1 also.
+ </para>
+ <para>
+ If the <literal>len</literal> field has the value -1, then
+ <literal>record</literal> is assumed to point to a constructed data
+ type. The <literal>format</literal> field will be used to determine
+ which encoder should be used to serialize the data.
+ </para>
+ <note>
+ <para>
+ If your backend generates structured records, it should use
+ <function>odr_malloc()</function> on the provided stream for allocating
+ data: This allows the frontend server to keep track of the record sizes.
+ </para>
+ </note>
+ <para>
+ The <literal>format</literal> field is mapped to an object identifier
+ in the direct reference of the resulting EXTERNAL representation
+ of the record.
+ </para>
+ <note>
+ <para>
+ The current version of &yaz; only supports the direct reference mode.
+ </para>
+ </note>
+ <synopsis>
+int (*bend_present) (void *handle, bend_present_rr *rr);
+
+typedef struct {
+ char *setname; /* set name */
+ int start;
+ int number; /* record number */
+ Odr_oid *format; /* format, transfer syntax (OID) */
+ Z_ReferenceId *referenceId;/* reference ID */
+ Z_RecordComposition *comp; /* Formatting instructions */
+ ODR stream; /* encoding stream - memory source if required */
+ ODR print; /* printing stream */
+ bend_request request;
+ bend_association association;
+
+ int hits; /* number of hits */
+ int errcode; /* 0==OK */
+ char *errstring; /* system error string or NULL */
+} bend_present_rr;
+ </synopsis>
+ <para>
+ The <function>bend_present</function> handler is called when
+ the server receives a Z39.50 Present Request.
+ The <literal>setname</literal>,
+ <literal>start</literal> and <literal>number</literal> is the
+ name of the result set - start position - and number of records to
+ be retrieved respectively. <literal>format</literal> and
+ <literal>comp</literal> is the preferred transfer syntax and element
+ specifications of the present request.
+ </para>
+ <para>
+ Note that this is handler serves as a supplement for
+ <function>bend_fetch</function> and need not to be defined in order to
+ support search - and retrieve.
+ </para>
+ </sect2>
+ <sect2 id="server.delete">
+ <title>Delete</title>
+ <para>
+ For back-ends that supports delete of a result set only one handler
+ must be defined.
+ </para>
+ <synopsis>
+int (*bend_delete)(void *handle, bend_delete_rr *rr);
+
+typedef struct bend_delete_rr {
+ int function;
+ int num_setnames;
+ char **setnames;
+ Z_ReferenceId *referenceId;
+ int delete_status; /* status for the whole operation */
+ int *statuses; /* status each set - indexed as setnames */
+ ODR stream;
+ ODR print;
+} bend_delete_rr;
+ </synopsis>
+ <note>
+ <para>
+ The delete set function definition is rather primitive, mostly because
+ we have had no practical need for it as of yet. If someone wants
+ to provide a full delete service, we'd be happy to add the
+ extra parameters that are required. Are there clients out there
+ that will actually delete sets they no longer need?
+ </para>
+ </note>
+ </sect2>
+ <sect2 id="server.scan">
+ <title>Scan</title>
+ <para>
+ For servers that wish to offer the scan service one handler
+ must be defined.
+ </para>
+ <synopsis>
+int (*bend_scan)(void *handle, bend_scan_rr *rr);
+
+typedef enum {
+ BEND_SCAN_SUCCESS, /* ok */
+ BEND_SCAN_PARTIAL /* not all entries could be found */
+} bend_scan_status;
+
+typedef struct bend_scan_rr {
+ int num_bases; /* number of elements in databaselist */
+ char **basenames; /* databases to search */
+ Odr_oid *attributeset;
+ Z_ReferenceId *referenceId; /* reference ID */
+ Z_AttributesPlusTerm *term;
+ ODR stream; /* encoding stream - memory source if required */
+ ODR print; /* printing stream */
+
+ int *step_size; /* step size */
+ int term_position; /* desired index of term in result list/returned */
+ int num_entries; /* number of entries requested/returned */
+
+ /* scan term entries. The called handler does not have
+ to allocate this. Size of entries is num_entries (see above) */
+ struct scan_entry *entries;
+ bend_scan_status status;
+ int errcode;
+ char *errstring;
+ char *scanClause; /* CQL scan clause */
+ char *setname; /* Scan in result set (NULL if omitted) */
+} bend_scan_rr;
+ </synopsis>
+ <para>
+ This backend server handles both Z39.50 scan
+ and SRU scan. In order for a handler to distinguish between SRU (CQL) scan
+ Z39.50 Scan , it must check for a non-NULL value of
+ <literal>scanClause</literal>.
+ </para>
+ <note>
+ <para>
+ if designed today, it would be a choice using a union or similar,
+ but that would break binary compatibility with existing servers.
+ </para>
+ </note>
+ </sect2>
+ </sect1>
+ <sect1 id="server.invocation">
+ <title>Application Invocation</title>
+ <para>
+ The finished application has the following
+ invocation syntax (by way of <function>statserv_main()</function>):
+ </para>
+ &gfs-synopsis;
+ <para>
+ The options are:
+ &gfs-options;
+ </para>
+ <para>
+ A listener specification consists of a transport mode followed by a
+ colon (:) followed by a listener address. The transport mode is
+ either <literal>tcp</literal>, <literal>unix:</literal> or
+ <literal>ssl</literal>.
+ </para>
+ <para>
+ For TCP and SSL, an address has the form
+ </para>
+ <synopsis>
+ hostname | IP-number [: portnumber]
+ </synopsis>
+ <para>
+ The port number defaults to 210 (standard Z39.50 port).
+ </para>
+ <para>
+ For UNIX, the address is the filename of socket.
+ </para>
+ <para>
+ For TCP/IP and SSL, the special hostnames <literal>@</literal>,
+ maps to <literal>IN6ADDR_ANY_INIT</literal> with
+ IPV4 binding as well (bindv6only=0),
+ The special hostname <literal>@4</literal> binds to
+ <literal>INADDR_ANY</literal> (IPV4 only listener).
+ The special hostname <literal>@6</literal> binds to
+ <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
+ </para>
+ <example id="server.example.running.unix">
+ <title>Running the GFS on Unix</title>
+ <para>
+ Assuming the server application <replaceable>appname</replaceable> is
+ started as root, the following will make it listen on port 210.
+ The server will change identity to <literal>nobody</literal>
+ and write its log to <filename>/var/log/app.log</filename>.
+ <screen>
+ application -l /var/log/app.log -u nobody tcp:@:210
+ </screen>
+ </para>
+ <para>
+ The server will accept Z39.50 requests and offer SRU service on port 210.
+ </para>
+ </example>
+ <example id="server.example.apache.sru">
+ <title>Setting up Apache as SRU Frontend</title>
+ <para>
+ If you use <ulink url="&url.apache;">Apache</ulink>
+ as your public web server and want to offer HTTP port 80
+ access to the YAZ server on 210, you can use the
+ <ulink url="&url.apache.directive.proxypass;">
+ <literal>ProxyPass</literal></ulink>
+ directive.
+ If you have virtual host
+ <literal>srw.mydomain</literal> you can use the following directives
+ in Apache's httpd.conf:
+ <screen>
+ <VirtualHost *>
+ ErrorLog /home/srw/logs/error_log
+ TransferLog /home/srw/logs/access_log
+ ProxyPass / http://srw.mydomain:210/
+ </VirtualHost>
+ </screen>
+ </para>
+ <para>
+ The above for the Apache 1.3 series.
+ </para>
+ </example>
+ <example id="server.example.local.access">
+ <title>Running a server with local access only</title>
+ <para>
+ Servers that is only being accessed from the local host should listen
+ on UNIX file socket rather than a Internet socket. To listen on
+ <filename>/tmp/mysocket</filename> start the server as follows:
+ <screen>
+ application unix:/tmp/mysocket
+ </screen>
+ </para>
+ </example>
+ </sect1>
+ <sect1 id="server.vhosts">
+ <title>GFS Configuration and Virtual Hosts</title>
+ &gfs-virtual;
+ </sect1>
+ </chapter>
+ <chapter id="asn">
+ <title>The Z39.50 ASN.1 Module</title>
+ <sect1 id="asn.introduction">
+ <title>Introduction</title>
+ <para>
+ The &asn; module provides you with a set of C struct definitions for the
+ various PDUs of the Z39.50 protocol, as well as for the complex types
+ appearing within the PDUs. For the primitive data types, the C
+ representation often takes the form of an ordinary C language type,
+ such as <literal>Odr_int</literal> which is equivalent to an integral
+ C integer. For ASN.1 constructs that have no direct
+ representation in C, such as general octet strings and bit strings,
+ the &odr; module (see section <link linkend="odr">The ODR Module</link>)
+ provides auxiliary definitions.
+ </para>
+ <para>
+ The &asn; module is located in sub directory <filename>z39.50</filename>.
+ There you'll find C files that implements encoders and decoders for the
+ Z39.50 types. You'll also find the protocol definitions:
+ <filename>z3950v3.asn</filename>, <filename>esupdate.asn</filename>,
+ and others.
+ </para>
+ </sect1>
+ <sect1 id="asn.preparing">
+ <title>Preparing PDUs</title>
+ <para>
+ A structure representing a complex ASN.1 type doesn't in itself contain the
+ members of that type. Instead, the structure contains
+ <emphasis>pointers</emphasis> to the members of the type.
+ This is necessary, in part, to allow a mechanism for specifying which
+ of the optional structure (SEQUENCE) members are present, and which
+ are not. It follows that you will need to somehow provide space for
+ the individual members of the structure, and set the pointers to
+ refer to the members.
+ </para>
+ <para>
+ The conversion routines don't care how you allocate and maintain your
+ C structures - they just follow the pointers that you provide.
+ Depending on the complexity of your application, and your personal
+ taste, there are at least three different approaches that you may take
+ when you allocate the structures.
+ </para>
+ <para>
+ You can use static or automatic local variables in the function that
+ prepares the PDU. This is a simple approach, and it provides the most
+ efficient form of memory management. While it works well for flat
+ PDUs like the InitReqest, it will generally not be sufficient for say,
+ the generation of an arbitrarily complex RPN query structure.
+ </para>
+ <para>
+ You can individually create the structure and its members using the
+ <function>malloc(2)</function> function. If you want to ensure that
+ the data is freed when it is no longer needed, you will have to
+ define a function that individually releases each member of a
+ structure before freeing the structure itself.
+ </para>
+ <para>
+ You can use the <function>odr_malloc()</function> function (see
+ <xref linkend="odr.use"/> for details). When you use
+ <function>odr_malloc()</function>, you can release all of the
+ allocated data in a single operation, independent of any pointers and
+ relations between the data. <function>odr_malloc()</function> is based on a
+ "nibble-memory"
+ scheme, in which large portions of memory are allocated, and then
+ gradually handed out with each call to <function>odr_malloc()</function>.
+ The next time you call <function>odr_reset()</function>, all of the
+ memory allocated since the last call is recycled for future use (actually,
+ it is placed on a free-list).
+ </para>
+ <para>
+ You can combine all of the methods described here. This will often be
+ the most practical approach. For instance, you might use
+ <function>odr_malloc()</function> to allocate an entire structure and
+ some of its elements, while you leave other elements pointing to global
+ or per-session default variables.
+ </para>
+ <para>
+ The &asn; module provides an important aid in creating new PDUs. For
+ each of the PDU types (say, <function>Z_InitRequest</function>), a
+ function is provided that allocates and initializes an instance of
+ that PDU type for you. In the case of the InitRequest, the function is
+ simply named <function>zget_InitRequest()</function>, and it sets up
+ reasonable default value for all of the mandatory members. The optional
+ members are generally initialized to null pointers. This last aspect
+ is very important: it ensures that if the PDU definitions are
+ extended after you finish your implementation (to accommodate
+ new versions of the protocol, say), you won't get into trouble with
+ uninitialized pointers in your structures. The functions use
+ <function>odr_malloc()</function> to
+ allocate the PDUs and its members, so you can free everything again with a
+ single call to <function>odr_reset()</function>. We strongly recommend
+ that you use the <literal>zget_*</literal>
+ functions whenever you are preparing a PDU (in a C++ API, the
+ <literal>zget_</literal>
+ functions would probably be promoted to constructors for the
+ individual types).
+ </para>
+ <para>
+ The prototype for the individual PDU types generally look like this:
+ </para>
+ <synopsis>
+ Z_<type> *zget_<type>(ODR o);
+ </synopsis>
+ <para>
+ eg.:
+ </para>
+ <synopsis>
+ Z_InitRequest *zget_InitRequest(ODR o);
+ </synopsis>
+ <para>
+ The &odr; handle should generally be your encoding stream, but it
+ needn't be.
+ </para>
+ <para>
+ As well as the individual PDU functions, a function
+ <function>zget_APDU()</function> is provided, which allocates
+ a top-level Z-APDU of the type requested:
+ </para>
+ <synopsis>
+ Z_APDU *zget_APDU(ODR o, int which);
+ </synopsis>
+ <para>
+ The <varname>which</varname> parameter is (of course) the discriminator
+ belonging to the <varname>Z_APDU</varname> <literal>CHOICE</literal> type.
+ All of the interface described here is provided by the &asn; module, and
+ you access it through the <filename>proto.h</filename> header file.
+ </para>
+ </sect1>
+ <sect1 id="asn.external">
+ <title>EXTERNAL Data</title>
+ <para>
+ In order to achieve extensibility and adaptability to different
+ application domains, the new version of the protocol defines many
+ structures outside of the main ASN.1 specification, referencing them
+ through ASN.1 EXTERNAL constructs. To simplify the construction and
+ access to the externally referenced data, the &asn; module defines a
+ specialized version of the EXTERNAL construct, called
+ <literal>Z_External</literal>.It is defined thus:
+ </para>
+ <screen>
+typedef struct Z_External
+{
+ Odr_oid *direct_reference;
+ int *indirect_reference;
+ char *descriptor;
+ enum
+ {
+ /* Generic types */
+ Z_External_single = 0,
+ Z_External_octet,
+ Z_External_arbitrary,
+
+ /* Specific types */
+ Z_External_SUTRS,
+ Z_External_explainRecord,
+ Z_External_resourceReport1,
+ Z_External_resourceReport2
+
+ ...
+
+ } which;
+ union
+ {
+ /* Generic types */
+ Odr_any *single_ASN1_type;
+ Odr_oct *octet_aligned;
+ Odr_bitmask *arbitrary;
+
+ /* Specific types */
+ Z_SUTRS *sutrs;
+ Z_ExplainRecord *explainRecord;
+ Z_ResourceReport1 *resourceReport1;
+ Z_ResourceReport2 *resourceReport2;
+
+ ...
+
+ } u;
+} Z_External;
+ </screen>
+ <para>
+ When decoding, the &asn; module will attempt to determine which
+ syntax describes the data by looking at the reference fields
+ (currently only the direct-reference). For ASN.1 structured data, you
+ need only consult the <literal>which</literal> field to determine the
+ type of data. You can the access the data directly through the union.
+ When constructing data for encoding, you set the union pointer to point
+ to the data, and set the <literal>which</literal> field accordingly.
+ Remember also to set the direct (or indirect) reference to the correct
+ OID for the data type.
+ For non-ASN.1 data such as MARC records, use the
+ <literal>octet_aligned</literal> arm of the union.
+ </para>
+ <para>
+ Some servers return ASN.1 structured data values (eg. database
+ records) as BER-encoded records placed in the
+ <literal>octet-aligned</literal> branch of the EXTERNAL CHOICE.
+ The ASN-module will <emphasis>not</emphasis> automatically decode
+ these records. To help you decode the records in the application, the
+ function
+ </para>
+ <screen>
+ Z_ext_typeent *z_ext_gettypebyref(const oid *oid);
+ </screen>
+ <para>
+ Can be used to retrieve information about the known, external data
+ types. The function return a pointer to a static area, or NULL, if no
+ match for the given direct reference is found. The
+ <literal>Z_ext_typeent</literal>
+ is defined as:
+ </para>
+ <screen>
+typedef struct Z_ext_typeent
+{
+ int oid[OID_SIZE]; /* the direct-reference OID. */
+ int what; /* discriminator value for the external CHOICE */
+ Odr_fun fun; /* decoder function */
+} Z_ext_typeent;
+ </screen>
+ <para>
+ The <literal>what</literal> member contains the
+ <literal>Z_External</literal> union discriminator value for the
+ given type: For the SUTRS record syntax, the value would be
+ <literal>Z_External_sutrs</literal>.
+ The <literal>fun</literal> member contains a pointer to the
+ function which encodes/decodes the given type. Again, for the SUTRS
+ record syntax, the value of <literal>fun</literal> would be
+ <literal>z_SUTRS</literal> (a function pointer).
+ </para>
+ <para>
+ If you receive an EXTERNAL which contains an octet-string value that
+ you suspect of being an ASN.1-structured data value, you can use
+ <literal>z_ext_gettypebyref</literal> to look for the provided
+ direct-reference.
+ If the return value is different from NULL, you can use the provided
+ function to decode the BER string (see <xref linkend="odr.use"/>
+ ).
+ </para>
+ <para>
+ If you want to <emphasis>send</emphasis> EXTERNALs containing
+ ASN.1-structured values in the occtet-aligned branch of the CHOICE, this
+ is possible too. However, on the encoding phase, it requires a somewhat
+ involved juggling around of the various buffers involved.
+ </para>
+ <para>
+ If you need to add new, externally defined data types, you must update
+ the struct above, in the source file <filename>prt-ext.h</filename>, as
+ well as the encoder/decoder in the file <filename>prt-ext.c</filename>.
+ When changing the latter, remember to update both the
+ <literal>arm</literal> arrary and the list
+ <literal>type_table</literal>, which drives the CHOICE biasing that
+ is necessary to tell the different, structured types apart
+ on decoding.
+ </para>
+ <note>
+ <para>
+ Eventually, the EXTERNAL processing will most likely
+ automatically insert the correct OIDs or indirect-refs. First,
+ however, we need to determine how application-context management
+ (specifically the presentation-context-list) should fit into the
+ various modules.
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="asn.pdu">
+ <title>PDU Contents Table</title>
+ <para>
+ We include, for reference, a listing of the fields of each top-level
+ PDU, as well as their default settings.
+ </para>
+ <table frame="top" id="asn.default.initialize.request">
+ <title>Default settings for PDU Initialize Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ idAuthentication</entry><entry>Z_IdAuthentication</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ implementationId</entry><entry>char*</entry><entry>"81"
+ </entry></row>
+ <row><entry>
+ implementationName</entry><entry>char*</entry><entry>"YAZ"
+ </entry></row>
+ <row><entry>
+ implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
+ </entry></row>
+ <row><entry>
+ userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.initialize.response">
+ <title>Default settings for PDU Initialize Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ result</entry><entry>Odr_bool</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ implementationId</entry><entry>char*</entry><entry>"id)"
+ </entry></row>
+ <row><entry>
+ implementationName</entry><entry>char*</entry><entry>"YAZ"
+ </entry></row>
+ <row><entry>
+ implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
+ </entry></row>
+ <row><entry>
+ userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.search.request">
+ <title>Default settings for PDU Search Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ smallSetUpperBound</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ largeSetLowerBound</entry><entry>Odr_int</entry><entry>1
+ </entry></row>
+ <row><entry>
+ mediumSetPresentNumber</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ replaceIndicator</entry><entry>Odr_bool</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ resultSetName</entry><entry>char *</entry><entry>"default"
+ </entry></row>
+ <row><entry>
+ num_databaseNames</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ databaseNames</entry><entry>char **</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ smallSetElementSetNames</entry><entry>Z_ElementSetNames
+ </entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ mediumSetElementSetNames</entry><entry>Z_ElementSetNames
+ </entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ query</entry><entry>Z_Query</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ additionalSearchInfo</entry><entry>Z_OtherInformation
+ </entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.search.response">
+ <title>Default settings for PDU Search Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resultCount</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ searchStatus</entry><entry>Odr_bool</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ resultSetStatus</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ presentStatus</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ records</entry><entry>Z_Records</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ additionalSearchInfo</entry>
+ <entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.present.request">
+ <title>Default settings for PDU Present Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resultSetId</entry><entry>char*</entry><entry>"default"
+ </entry></row>
+ <row><entry>
+ resultSetStartPoint</entry><entry>Odr_int</entry><entry>1
+ </entry></row>
+ <row><entry>
+ numberOfRecordsRequested</entry><entry>Odr_int</entry><entry>10
+ </entry></row>
+ <row><entry>
+ num_ranges</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ additionalRanges</entry><entry>Z_Range</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ recordComposition</entry><entry>Z_RecordComposition</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ maxSegmentCount</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ maxRecordSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ maxSegmentSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.present.response">
+ <title>Default settings for PDU Present Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ presentStatus</entry><entry>Odr_int</entry><entry>Z_PresentStatus_success
+ </entry></row>
+ <row><entry>
+ records</entry><entry>Z_Records</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.delete.result.set.request">
+ <title>Default settings for Delete Result Set Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>referenceId
+ </entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ deleteFunction</entry><entry>Odr_int</entry><entry>Z_DeleteResultSetRequest_list
+ </entry></row>
+ <row><entry>
+ num_ids</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ resultSetList</entry><entry>char**</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.delete.result.set.response">
+ <title>Default settings for Delete Result Set Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ deleteOperationStatus</entry><entry>Odr_int</entry>
+ <entry>Z_DeleteStatus_success</entry></row>
+ <row><entry>
+ num_statuses</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ deleteListStatuses</entry><entry>Z_ListStatus**</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberNotDeleted</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ num_bulkStatuses</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ bulkStatuses</entry><entry>Z_ListStatus</entry><entry>NUL
+ L</entry></row>
+ <row><entry>
+ deleteMessage</entry><entry>char*</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.scan.request">
+ <title>Default settings for Scan Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ num_databaseNames</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ databaseNames</entry><entry>char**</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ attributeSet</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ termListAndStartPoint</entry><entry>Z_AttributesPlus...
+ </entry><entry>NULL</entry></row>
+ <row><entry>
+ stepSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberOfTermsRequested</entry><entry>Odr_int</entry><entry>20
+ </entry></row>
+ <row><entry>
+ preferredPositionInResponse</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.scan.response">
+ <title>Default settings for Scan Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ stepSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ scanStatus</entry><entry>Odr_int</entry><entry>Z_Scan_success
+ </entry></row>
+ <row><entry>
+ numberOfEntriesReturned</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ positionOfTerm</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ entries</entry><entry>Z_ListEntris</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ attributeSet</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.trigger.resource.control.request">
+ <title>Default settings for Trigger Resource Control Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ requestedAction</entry><entry>Odr_int</entry><entry>
+ Z_TriggerResourceCtrl_resou..
+ </entry></row>
+ <row><entry>
+ prefResourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resultSetWanted</entry><entry>Odr_bool</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.resource.control.request">
+ <title>Default settings for Resource Control Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ suspendedFlag</entry><entry>Odr_bool</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resourceReport</entry><entry>Z_External</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ partialResultsAvailable</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ responseRequired</entry><entry>Odr_bool</entry><entry>FALSE
+ </entry></row>
+ <row><entry>
+ triggeredRequestFlag</entry><entry>Odr_bool</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.resource.control.response">
+ <title>Default settings for Resource Control Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ continueFlag</entry><entry>bool_t</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ resultSetWanted</entry><entry>bool_t</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.access.control.request">
+ <title>Default settings for Access Control Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ which</entry><entry>enum</entry><entry>Z_AccessRequest_simpleForm;
+ </entry></row>
+ <row><entry>
+ u</entry><entry>union</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.access.control.response">
+ <title>Default settings for Access Control Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ which</entry><entry>enum</entry><entry>Z_AccessResponse_simpleForm
+ </entry></row>
+ <row><entry>
+ u</entry><entry>union</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ diagnostic</entry><entry>Z_DiagRec</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.segment">
+ <title>Default settings for Segment</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>value=0
+ </entry></row>
+ <row><entry>
+ num_segmentRecords</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ segmentRecords</entry><entry>Z_NamePlusRecord</entry><entry>NULL
+ </entry></row>
+ <row><entry>otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.close">
+ <title>Default settings for Close</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ closeReason</entry><entry>Odr_int</entry><entry>Z_Close_finished
+ </entry></row>
+ <row><entry>
+ diagnosticInformation</entry><entry>char*</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resourceFormat</entry><entry>Z_External</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ </chapter>
+ <chapter id="soap">
+ <title>SOAP and SRU</title>
+ <sect1 id="soap.introduction">
+ <title>Introduction</title>
+ <para>
+ &yaz; uses a very simple implementation of
+ <ulink url="&url.soap;">SOAP</ulink> that only,
+ currenly, supports what is sufficient to offer SRU SOAP functionality.
+ The implementation uses the
+ <ulink url="&url.libxml2.api.tree;">tree API</ulink> of
+ libxml2 to encode and decode SOAP packages.
+ </para>
+ <para>
+ Like the Z39.50 ASN.1 module, the &yaz; SRU implementation uses
+ simple C structs to represent SOAP packages as well as
+ HTTP packages.
+ </para>
+ </sect1>
+ <sect1 id="soap.http">
+ <title>HTTP</title>
+ <para>
+ &yaz; only offers HTTP as transport carrier for SOAP, but it is
+ relatively easy to change that.
+ </para>
+ <para>
+ The following definition of <literal>Z_GDU</literal> (Generic Data
+ Unit) allows for both HTTP and Z39.50 in one packet.
+ </para>
+ <synopsis>
+#include <yaz/zgdu.h>
+
+#define Z_GDU_Z3950 1
+#define Z_GDU_HTTP_Request 2
+#define Z_GDU_HTTP_Response 3
+typedef struct {
+ int which;
+ union {
+ Z_APDU *z3950;
+ Z_HTTP_Request *HTTP_Request;
+ Z_HTTP_Response *HTTP_Response;
+ } u;
+} Z_GDU ;
+ </synopsis>
+ <para>
+ The corresponding Z_GDU encoder/decoder is <function>z_GDU</function>.
+ The <literal>z3950</literal> is any of the known BER encoded Z39.50
+ APDUs.
+ <literal>HTTP_Request</literal> and <literal>HTTP_Response</literal>
+ is the HTTP Request and Response respectively.
+ </para>
+ </sect1>
+ <sect1 id="soap.xml">
+ <title>SOAP Packages</title>
+ <para>
+ Every SOAP package in &yaz; is represented as follows:
+ <synopsis>
+#include <yaz/soap.h>
+
+typedef struct {
+ char *fault_code;
+ char *fault_string;
+ char *details;
+} Z_SOAP_Fault;
+
+typedef struct {
+ int no;
+ char *ns;
+ void *p;
+} Z_SOAP_Generic;
+
+#define Z_SOAP_fault 1
+#define Z_SOAP_generic 2
+#define Z_SOAP_error 3
+typedef struct {
+ int which;
+ union {
+ Z_SOAP_Fault *fault;
+ Z_SOAP_Generic *generic;
+ Z_SOAP_Fault *soap_error;
+ } u;
+ const char *ns;
+} Z_SOAP;
+ </synopsis>
+ </para>
+ <para>
+ The <literal>fault</literal> and <literal>soap_error</literal>
+ arms represent both a SOAP fault - struct
+ <literal>Z_SOAP_Fault</literal>. Any other generic
+ (valid) package is represented by <literal>Z_SOAP_Generic</literal>.
+ </para>
+ <para>
+ The <literal>ns</literal> as part of <literal>Z_SOAP</literal>
+ is the namespace for SOAP itself and reflects the SOAP
+ version. For version 1.1 it is
+ <literal>http://schemas.xmlsoap.org/soap/envelope/</literal>,
+ for version 1.2 it is
+ <literal>http://www.w3.org/2001/06/soap-envelope</literal>.
+ </para>
+ <synopsis>
+int z_soap_codec(ODR o, Z_SOAP **pp,
+ char **content_buf, int *content_len,
+ Z_SOAP_Handler *handlers);
+ </synopsis>
+ <para>
+ The <literal>content_buf</literal> and <literal>content_len</literal>
+ is XML buffer and length of buffer respectively.
+ </para>
+ <para>
+ The <literal>handlers</literal> is a list of SOAP codec
+ handlers - one handler for each service namespace. For SRU SOAP, the
+ namespace would be <literal>http://www.loc.gov/zing/srw/v1.0/</literal>.
+ </para>
+ <para>
+ When decoding, the <function>z_soap_codec</function>
+ inspects the XML content
+ and tries to match one of the services namespaces of the
+ supplied handlers. If there is a match a handler function
+ is invoked which decodes that particular SOAP package.
+ If successful, the returned <literal>Z_SOAP</literal> package will be
+ of type <literal>Z_SOAP_Generic</literal>.
+ Member <literal>no</literal> is
+ set the offset of handler that matched; <literal>ns</literal>
+ is set to namespace of matching handler; the void pointer
+ <literal>p</literal> is set to the C data structure assocatiated
+ with the handler.
+ </para>
+ <para>
+ When a NULL namespace is met (member <literal>ns</literal> bwlow),
+ that specifies end-of-list.
+ </para>
+ <para>
+ Each handler is defined as follows:
+ <synopsis>
+typedef struct {
+ char *ns;
+ void *client_data;
+ Z_SOAP_fun f;
+} Z_SOAP_Handler;
+ </synopsis>
+ The <literal>ns</literal> is namespace of service associated with
+ handler <literal>f</literal>. <literal>client_data</literal>
+ is user-defined data which is passed to handler.
+ </para>
+ <para>
+ The prototype for a SOAP service handler is:
+ <synopsis>
+int handler(ODR o, void * ptr, void **handler_data,
+ void *client_data, const char *ns);
+ </synopsis>
+ The <parameter>o</parameter> specifies the mode (decode/encode)
+ as usual. The second argument, <parameter>ptr</parameter>,
+ is a libxml2 tree node pointer (<literal>xmlNodePtr</literal>)
+ and is a pointer to the <literal>Body</literal> element
+ of the SOAP package. The <parameter>handler_data</parameter>
+ is an opaque pointer to a C definitions associated with the
+ SOAP service. <parameter>client_data</parameter> is the pointer
+ which was set as part of the <literal>Z_SOAP_handler</literal>.
+ Finally, <parameter>ns</parameter> the service namespace.
+ </para>
+ </sect1>
+ <sect1 id="soap.srw">
+ <title>SRU</title>
+ <para>
+ SRU SOAP is just one implementation of a SOAP handler as described
+ in the previous section.
+ The encoder/decoder handler for SRU is defined as
+ follows:
+ <synopsis>
+#include <yaz/srw.h>
+
+int yaz_srw_codec(ODR o, void * pptr,
+ Z_SRW_GDU **handler_data,
+ void *client_data, const char *ns);
+ </synopsis>
+ Here, <literal>Z_SRW_GDU</literal> is either
+ searchRetrieveRequest or a searchRetrieveResponse.
+ </para>
+ <note>
+ <para>
+ The xQuery and xSortKeys are not handled yet by
+ the SRW implementation of &yaz;. Explain is also missing.
+ Future versions of &yaz; will include these features.
+ </para>
+ </note>
+ <para>
+ The definition of searchRetrieveRequest is:
+ <synopsis>
+typedef struct {
+
+#define Z_SRW_query_type_cql 1
+#define Z_SRW_query_type_xcql 2
+#define Z_SRW_query_type_pqf 3
+ int query_type;
+ union {
+ char *cql;
+ char *xcql;
+ char *pqf;
+ } query;
+
+#define Z_SRW_sort_type_none 1
+#define Z_SRW_sort_type_sort 2
+#define Z_SRW_sort_type_xSort 3
+ int sort_type;
+ union {
+ char *none;
+ char *sortKeys;
+ char *xSortKeys;
+ } sort;
+ int *startRecord;
+ int *maximumRecords;
+ char *recordSchema;
+ char *recordPacking;
+ char *database;
+} Z_SRW_searchRetrieveRequest;
+ </synopsis>
+ Please observe that data of type xsd:string is represented
+ as a char pointer (<literal>char *</literal>). A null pointer
+ means that the element is absent.
+ Data of type xsd:integer is representd as a pointer to
+ an int (<literal>int *</literal>). Again, a null pointer
+ us used for absent elements.
+ </para>
+ <para>
+ The SearchRetrieveResponse has the following definition.
+ <synopsis>
+typedef struct {
+ int * numberOfRecords;
+ char * resultSetId;
+ int * resultSetIdleTime;
+
+ Z_SRW_record *records;
+ int num_records;
+
+ Z_SRW_diagnostic *diagnostics;
+ int num_diagnostics;
+ int *nextRecordPosition;
+} Z_SRW_searchRetrieveResponse;
+ </synopsis>
+ The <literal>num_records</literal> and <literal>num_diagnostics</literal>
+ is number of returned records and diagnostics respectively and also
+ correspond to the "size of" arrays <literal>records</literal>
+ and <literal>diagnostics</literal>.
+ </para>
+ <para>
+ A retrieval record is defined as follows:
+ <synopsis>
+typedef struct {
+ char *recordSchema;
+ char *recordData_buf;
+ int recordData_len;
+ int *recordPosition;
+} Z_SRW_record;
+ </synopsis>
+ The record data is defined as a buffer of some length so that
+ data can be of any type. SRW 1.0 currenly doesn't allow for this
+ (only XML), but future versions might do.
+ </para>
+ <para>
+ And, a diagnostic as:
+ <synopsis>
+typedef struct {
+ int *code;
+ char *details;
+} Z_SRW_diagnostic;
+ </synopsis>
+ </para>
+ </sect1>
+ </chapter>
+ <chapter id="tools">
+ <title>Supporting Tools</title>
+ <para>
+ In support of the service API - primarily the ASN module, which
+ provides the pro-grammatic interface to the Z39.50 APDUs, &yaz; contains
+ a collection of tools that support the development of applications.
+ </para>
+ <sect1 id="tools.query">
+ <title>Query Syntax Parsers</title>
+ <para>
+ Since the type-1 (RPN) query structure has no direct, useful string
+ representation, every origin application needs to provide some form of
+ mapping from a local query notation or representation to a
+ <token>Z_RPNQuery</token> structure. Some programmers will prefer to
+ construct the query manually, perhaps using
+ <function>odr_malloc()</function> to simplify memory management.
+ The &yaz; distribution includes three separate, query-generating tools
+ that may be of use to you.
+ </para>
+ <sect2 id="PQF">
+ <title>Prefix Query Format</title>
+ <para>
+ Since RPN or reverse polish notation is really just a fancy way of
+ describing a suffix notation format (operator follows operands), it
+ would seem that the confusion is total when we now introduce a prefix
+ notation for RPN. The reason is one of simple laziness - it's somewhat
+ simpler to interpret a prefix format, and this utility was designed
+ for maximum simplicity, to provide a baseline representation for use
+ in simple test applications and scripting environments (like Tcl). The
+ demonstration client included with YAZ uses the PQF.
+ </para>
+ <note>
+ <para>
+ The PQF have been adopted by other parties developing Z39.50
+ software. It is often referred to as Prefix Query Notation
+ - PQN.
+ </para>
+ </note>
+ <para>
+ The PQF is defined by the pquery module in the YAZ library.
+ There are two sets of function that have similar behavior. First
+ set operates on a PQF parser handle, second set doesn't. First set
+ set of functions are more flexible than the second set. Second set
+ is obsolete and is only provided to ensure backwards compatibility.
+ </para>
+ <para>
+ First set of functions all operate on a PQF parser handle:
+ </para>
+ <synopsis>
+ #include <yaz/pquery.h>
+
+ YAZ_PQF_Parser yaz_pqf_create(void);
+
+ void yaz_pqf_destroy(YAZ_PQF_Parser p);
+
+ Z_RPNQuery *yaz_pqf_parse(YAZ_PQF_Parser p, ODR o, const char *qbuf);
+
+ Z_AttributesPlusTerm *yaz_pqf_scan(YAZ_PQF_Parser p, ODR o,
+ Odr_oid **attributeSetId, const char *qbuf);
+
+ int yaz_pqf_error(YAZ_PQF_Parser p, const char **msg, size_t *off);
+ </synopsis>
+ <para>
+ A PQF parser is created and destructed by functions
+ <function>yaz_pqf_create</function> and
+ <function>yaz_pqf_destroy</function> respectively.
+ Function <function>yaz_pqf_parse</function> parses query given
+ by string <literal>qbuf</literal>. If parsing was successful,
+ a Z39.50 RPN Query is returned which is created using ODR stream
+ <literal>o</literal>. If parsing failed, a NULL pointer is
+ returned.
+ Function <function>yaz_pqf_scan</function> takes a scan query in
+ <literal>qbuf</literal>. If parsing was successful, the function
+ returns attributes plus term pointer and modifies
+ <literal>attributeSetId</literal> to hold attribute set for the
+ scan request - both allocated using ODR stream <literal>o</literal>.
+ If parsing failed, yaz_pqf_scan returns a NULL pointer.
+ Error information for bad queries can be obtained by a call to
+ <function>yaz_pqf_error</function> which returns an error code and
+ modifies <literal>*msg</literal> to point to an error description,
+ and modifies <literal>*off</literal> to the offset within last
+ query were parsing failed.
+ </para>
+ <para>
+ The second set of functions are declared as follows:
+ </para>
+ <synopsis>
+ #include <yaz/pquery.h>
+
+ Z_RPNQuery *p_query_rpn(ODR o, oid_proto proto, const char *qbuf);
+
+ Z_AttributesPlusTerm *p_query_scan(ODR o, oid_proto proto,
+ Odr_oid **attributeSetP, const char *qbuf);
+
+ int p_query_attset(const char *arg);
+ </synopsis>
+ <para>
+ The function <function>p_query_rpn()</function> takes as arguments an
+ &odr; stream (see section <link linkend="odr">The ODR Module</link>)
+ to provide a memory source (the structure created is released on
+ the next call to <function>odr_reset()</function> on the stream), a
+ protocol identifier (one of the constants <token>PROTO_Z3950</token> and
+ <token>PROTO_SR</token>), an attribute set reference, and
+ finally a null-terminated string holding the query string.
+ </para>
+ <para>
+ If the parse went well, <function>p_query_rpn()</function> returns a
+ pointer to a <literal>Z_RPNQuery</literal> structure which can be
+ placed directly into a <literal>Z_SearchRequest</literal>.
+ If parsing failed, due to syntax error, a NULL pointer is returned.
+ </para>
+ <para>
+ The <literal>p_query_attset</literal> specifies which attribute set
+ to use if the query doesn't specify one by the
+ <literal>@attrset</literal> operator.
+ The <literal>p_query_attset</literal> returns 0 if the argument is a
+ valid attribute set specifier; otherwise the function returns -1.
+ </para>
+ <para>
+ The grammar of the PQF is as follows:
+ </para>
+ <literallayout>
+ query ::= top-set query-struct.
+
+ top-set ::= [ '@attrset' string ]
+
+ query-struct ::= attr-spec | simple | complex | '@term' term-type query
+
+ attr-spec ::= '@attr' [ string ] string query-struct
+
+ complex ::= operator query-struct query-struct.
+
+ operator ::= '@and' | '@or' | '@not' | '@prox' proximity.
+
+ simple ::= result-set | term.
+
+ result-set ::= '@set' string.
+
+ term ::= string.
+
+ proximity ::= exclusion distance ordered relation which-code unit-code.
+
+ exclusion ::= '1' | '0' | 'void'.
+
+ distance ::= integer.
+
+ ordered ::= '1' | '0'.
+
+ relation ::= integer.
+
+ which-code ::= 'known' | 'private' | integer.
+
+ unit-code ::= integer.
+
+ term-type ::= 'general' | 'numeric' | 'string' | 'oid' | 'datetime' | 'null'.
+ </literallayout>
+ <para>
+ You will note that the syntax above is a fairly faithful
+ representation of RPN, except for the Attribute, which has been
+ moved a step away from the term, allowing you to associate one or more
+ attributes with an entire query structure. The parser will
+ automatically apply the given attributes to each term as required.
+ </para>
+ <para>
+ The @attr operator is followed by an attribute specification
+ (<literal>attr-spec</literal> above). The specification consists
+ of an optional attribute set, an attribute type-value pair and
+ a sub-query. The attribute type-value pair is packed in one string:
+ an attribute type, an equals sign, and an attribute value, like this:
+ <literal>@attr 1=1003</literal>.
+ The type is always an integer but the value may be either an
+ integer or a string (if it doesn't start with a digit character).
+ A string attribute-value is encoded as a Type-1 ``complex''
+ attribute with the list of values containing the single string
+ specified, and including no semantic indicators.
+ </para>
+ <para>
+ Version 3 of the Z39.50 specification defines various encoding of terms.
+ Use <literal>@term </literal> <replaceable>type</replaceable>
+ <replaceable>string</replaceable>,
+ where type is one of: <literal>general</literal>,
+ <literal>numeric</literal> or <literal>string</literal>
+ (for InternationalString).
+ If no term type has been given, the <literal>general</literal> form
+ is used. This is the only encoding allowed in both versions 2 and 3
+ of the Z39.50 standard.
+ </para>
+ <sect3 id="PQF-prox">
+ <title>Using Proximity Operators with PQF</title>
+ <note>
+ <para>
+ This is an advanced topic, describing how to construct
+ queries that make very specific requirements on the
+ relative location of their operands.
+ You may wish to skip this section and go straight to
+ <link linkend="pqf-examples">the example PQF queries</link>.
+ </para>
+ <para>
+ <warning>
+ <para>
+ Most Z39.50 servers do not support proximity searching, or
+ support only a small subset of the full functionality that
+ can be expressed using the PQF proximity operator. Be
+ aware that the ability to <emphasis>express</emphasis> a
+ query in PQF is no guarantee that any given server will
+ be able to <emphasis>execute</emphasis> it.
+ </para>
+ </warning>
+ </para>
+ </note>
+ <para>
+ The proximity operator <literal>@prox</literal> is a special
+ and more restrictive version of the conjunction operator
+ <literal>@and</literal>. Its semantics are described in
+ section 3.7.2 (Proximity) of Z39.50 the standard itself, which
+ can be read on-line at
+ <ulink url="&url.z39.50.proximity;"/>
+ </para>
+ <para>
+ In PQF, the proximity operation is represented by a sequence
+ of the form
+ <screen>
+ @prox <replaceable>exclusion</replaceable> <replaceable>distance</replaceable> <replaceable>ordered</replaceable> <replaceable>relation</replaceable> <replaceable>which-code</replaceable> <replaceable>unit-code</replaceable>
+ </screen>
+ in which the meanings of the parameters are as described in in
+ the standard, and they can take the following values:
+ <itemizedlist>
+ <listitem>
+ <formalpara><title>exclusion</title>
+ <para>
+ 0 = false (i.e. the proximity condition specified by the
+ remaining parameters must be satisfied) or
+ 1 = true (the proximity condition specified by the
+ remaining parameters must <emphasis>not</emphasis> be
+ satisifed).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>distance</title><para>
+ An integer specifying the difference between the locations
+ of the operands: e.g. two adjacent words would have
+ distance=1 since their locations differ by one unit.
+ </para>
+ </formalpara></listitem>
+ <listitem>
+ <formalpara><title>ordered</title><para>
+ 1 = ordered (the operands must occur in the order the
+ query specifies them) or
+ 0 = unordered (they may appear in either order).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>relation</title><para>
+ Recognised values are
+ 1 (lessThan),
+ 2 (lessThanOrEqual),
+ 3 (equal),
+ 4 (greaterThanOrEqual),
+ 5 (greaterThan) and
+ 6 (notEqual).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>which-code</title><para>
+ <literal>known</literal>
+ or
+ <literal>k</literal>
+ (the unit-code parameter is taken from the well-known list
+ of alternatives described in below) or
+ <literal>private</literal>
+ or
+ <literal>p</literal>
+ (the unit-code paramater has semantics specific to an
+ out-of-band agreement such as a profile).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>unit-code</title><para>
+ If the which-code parameter is <literal>known</literal>
+ then the recognised values are
+ 1 (character),
+ 2 (word),
+ 3 (sentence),
+ 4 (paragraph),
+ 5 (section),
+ 6 (chapter),
+ 7 (document),
+ 8 (element),
+ 9 (subelement),
+ 10 (elementType) and
+ 11 (byte).
+ If which-code is <literal>private</literal> then the
+ acceptable values are determined by the profile.
+ </para>
+ </formalpara>
+ </listitem>
+ </itemizedlist>
+ (The numeric values of the relation and well-known unit-code
+ parameters are taken straight from
+ <ulink url="&url.z39.50.proximity.asn1;"
+ >the ASN.1</ulink> of the proximity structure in the standard.)
+ </para>
+ </sect3>
+ <sect3 id="pqf-examples">
+ <title>PQF queries</title>
+ <example id="example.pqf.simple.terms">
+ <title>PQF queries using simple terms</title>
+ <para>
+ <screen>
+ dylan
+
+ "bob dylan"
+ </screen>
+ </para>
+ </example>
+ <example id="pqf.example.pqf.boolean.operators">
+ <title>PQF boolean operators</title>
+ <para>
+ <screen>
+ @or "dylan" "zimmerman"
+
+ @and @or dylan zimmerman when
+
+ @and when @or dylan zimmerman
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.result.sets">
+ <title>PQF references to result sets</title>
+ <para>
+ <screen>
+ @set Result-1
+
+ @and @set seta @set setb
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.attributes">
+ <title>Attributes for terms</title>
+ <para>
+ <screen>
+ @attr 1=4 computer
+
+ @attr 1=4 @attr 4=1 "self portrait"
+
+ @attrset exp1 @attr 1=1 CategoryList
+
+ @attr gils 1=2008 Copenhagen
+
+ @attr 1=/book/title computer
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.proximity">
+ <title>PQF Proximity queries</title>
+ <para>
+ <screen>
+ @prox 0 3 1 2 k 2 dylan zimmerman
+ </screen>
+ Here the parameters 0, 3, 1, 2, k and 2 represent exclusion,
+ distance, ordered, relation, which-code and unit-code, in that
+ order. So:
+ <itemizedlist>
+ <listitem>
+ <para>exclusion = 0: the proximity condition must hold</para>
+ </listitem>
+ <listitem>
+ <para>distance = 3: the terms must be three units apart</para>
+ </listitem>
+ <listitem>
+ <para>
+ ordered = 1: they must occur in the order they are specified
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ relation = 2: lessThanOrEqual (to the distance of 3 units)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ which-code is ``known'', so the standard unit-codes are used
+ </para>
+ </listitem>
+ <listitem>
+ <para>unit-code = 2: word.</para>
+ </listitem>
+ </itemizedlist>
+ So the whole proximity query means that the words
+ <literal>dylan</literal> and <literal>zimmerman</literal> must
+ both occur in the record, in that order, differing in position
+ by three or fewer words (i.e. with two or fewer words between
+ them.) The query would find ``Bob Dylan, aka. Robert
+ Zimmerman'', but not ``Bob Dylan, born as Robert Zimmerman''
+ since the distance in this case is four.
+ </para>
+ </example>
+ <example id="example.pqf.search.term.type">
+ <title>PQF specification of search term type</title>
+ <para>
+ <screen>
+ @term string "a UTF-8 string, maybe?"
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.mixed.queries">
+ <title>PQF mixed queries</title>
+ <para>
+ <screen>
+ @or @and bob dylan @set Result-1
+
+ @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
+
+ @and @attr 2=4 @attr gils 1=2038 -114 @attr 2=2 @attr gils 1=2039 -109
+ </screen>
+ The last of these examples is a spatial search: in
+ <ulink url="http://www.gils.net/prof_v2.html#sec_7_4"
+ >the GILS attribute set</ulink>,
+ access point
+ 2038 indicates West Bounding Coordinate and
+ 2030 indicates East Bounding Coordinate,
+ so the query is for areas extending from -114 degrees
+ to no more than -109 degrees.
+ </para>
+ </example>
+ </sect3>
+ </sect2>
+ <sect2 id="CCL"><title>CCL</title>
+ <para>
+ Not all users enjoy typing in prefix query structures and numerical
+ attribute values, even in a minimalistic test client. In the library
+ world, the more intuitive Common Command Language - CCL (ISO 8777)
+ has enjoyed some popularity - especially before the widespread
+ availability of graphical interfaces. It is still useful in
+ applications where you for some reason or other need to provide a
+ symbolic language for expressing boolean query structures.
+ </para>
+ <sect3 id="ccl.syntax">
+ <title>CCL Syntax</title>
+ <para>
+ The CCL parser obeys the following grammar for the FIND argument.
+ The syntax is annotated by in the lines prefixed by
+ <literal>--</literal>.
+ </para>
+ <screen>
+ CCL-Find ::= CCL-Find Op Elements
+ | Elements.
+
+ Op ::= "and" | "or" | "not"
+ -- The above means that Elements are separated by boolean operators.
+
+ Elements ::= '(' CCL-Find ')'
+ | Set
+ | Terms
+ | Qualifiers Relation Terms
+ | Qualifiers Relation '(' CCL-Find ')'
+ | Qualifiers '=' string '-' string
+ -- Elements is either a recursive definition, a result set reference, a
+ -- list of terms, qualifiers followed by terms, qualifiers followed
+ -- by a recursive definition or qualifiers in a range (lower - upper).
+
+ Set ::= 'set' = string
+ -- Reference to a result set
+
+ Terms ::= Terms Prox Term
+ | Term
+ -- Proximity of terms.
+
+ Term ::= Term string
+ | string
+ -- This basically means that a term may include a blank
+
+ Qualifiers ::= Qualifiers ',' string
+ | string
+ -- Qualifiers is a list of strings separated by comma
+
+ Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
+ -- Relational operators. This really doesn't follow the ISO8777
+ -- standard.
+
+ Prox ::= '%' | '!'
+ -- Proximity operator
+
+ </screen>
+ <example id="example.ccl.queries">
+ <title>CCL queries</title>
+ <para>
+ The following queries are all valid:
+ </para>
+ <screen>
+ dylan
+
+ "bob dylan"
+
+ dylan or zimmerman
+
+ set=1
+
+ (dylan and bob) or set=1
+
+ righttrunc?
+
+ "notrunc?"
+
+ singlechar#mask
+ </screen>
+ <para>
+ Assuming that the qualifiers <literal>ti</literal>,
+ <literal>au</literal>
+ and <literal>date</literal> are defined we may use:
+ </para>
+ <screen>
+ ti=self portrait
+
+ au=(bob dylan and slow train coming)
+
+ date>1980 and (ti=((self portrait)))
+ </screen>
+ </example>
+ </sect3>
+ <sect3 id="ccl.qualifiers">
+ <title>CCL Qualifiers</title>
+ <para>
+ Qualifiers are used to direct the search to a particular searchable
+ index, such as title (ti) and author indexes (au). The CCL standard
+ itself doesn't specify a particular set of qualifiers, but it does
+ suggest a few short-hand notations. You can customize the CCL parser
+ to support a particular set of qualifiers to reflect the current target
+ profile. Traditionally, a qualifier would map to a particular
+ use-attribute within the BIB-1 attribute set. It is also
+ possible to set other attributes, such as the structure
+ attribute.
+ </para>
+ <para>
+ A CCL profile is a set of predefined CCL qualifiers that may be
+ read from a file or set in the CCL API.
+ The YAZ client reads its CCL qualifiers from a file named
+ <filename>default.bib</filename>. There are four types of
+ lines in a CCL profile: qualifier specification,
+ qualifier alias, comments and directives.
+ </para>
+ <sect4 id="ccl.qualifier.specification">
+ <title>Qualifier specification</title>
+ <para>
+ A qualifier specification is of the form:
+ </para>
+ <para>
+ <replaceable>qualifier-name</replaceable>
+ [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable>
+ [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable> ...
+ </para>
+ <para>
+ where <replaceable>qualifier-name</replaceable> is the name of the
+ qualifier to be used (eg. <literal>ti</literal>),
+ <replaceable>type</replaceable> is attribute type in the attribute
+ set (Bib-1 is used if no attribute set is given) and
+ <replaceable>val</replaceable> is attribute value.
+ The <replaceable>type</replaceable> can be specified as an
+ integer or as it be specified either as a single-letter:
+ <literal>u</literal> for use,
+ <literal>r</literal> for relation,<literal>p</literal> for position,
+ <literal>s</literal> for structure,<literal>t</literal> for truncation
+ or <literal>c</literal> for completeness.
+ The attributes for the special qualifier name <literal>term</literal>
+ are used when no CCL qualifier is given in a query.
+ <table id="ccl.common.bib1.attributes">
+ <title>Common Bib-1 attributes</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="type"></colspec>
+ <colspec colwidth="9*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>u=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Use attribute (1). Common use attributes are
+ 1 Personal-name, 4 Title, 7 ISBN, 8 ISSN, 30 Date,
+ 62 Subject, 1003 Author), 1016 Any. Specify value
+ as an integer.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>r=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Relation attribute (2). Common values are
+ 1 <, 2 <=, 3 =, 4 >=, 5 >, 6 <>,
+ 100 phonetic, 101 stem, 102 relevance, 103 always matches.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>p=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Position attribute (3). Values: 1 first in field, 2
+ first in any subfield, 3 any position in field.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>s=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Structure attribute (4). Values: 1 phrase, 2 word,
+ 3 key, 4 year, 5 date, 6 word list, 100 date (un),
+ 101 name (norm), 102 name (un), 103 structure, 104 urx,
+ 105 free-form-text, 106 document-text, 107 local-number,
+ 108 string, 109 numeric string.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>t=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Truncation attribute (5). Values: 1 right, 2 left,
+ 3 left& right, 100 none, 101 process #, 102 regular-1,
+ 103 regular-2, 104 CCL.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>c=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Completeness attribute (6). Values: 1 incomplete subfield,
+ 2 complete subfield, 3 complete field.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </para>
+ <para>
+ Refer to <xref linkend="bib1"/> or the complete
+ <ulink url="&url.z39.50.attset.bib1;">list of Bib-1 attributes</ulink>
+ </para>
+ <para>
+ It is also possible to specify non-numeric attribute values,
+ which are used in combination with certain types.
+ The special combinations are:
+ <table id="ccl.special.attribute.combos">
+ <title>Special attribute combos</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="name"></colspec>
+ <colspec colwidth="9*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>s=pw</literal></entry>
+ <entry>
+ The structure is set to either word or phrase depending
+ on the number of tokens in a term (phrase-word).
+ </entry>
+ </row>
+ <row>
+ <entry><literal>s=al</literal></entry>
+ <entry>
+ Each token in the term is ANDed. (and-list).
+ This does not set the structure at all.
+ </entry>
+ </row>
+ <row><entry><literal>s=ol</literal></entry>
+ <entry>
+ Each token in the term is ORed. (or-list).
+ This does not set the structure at all.
+ </entry>
+ </row>
+ <row><entry><literal>s=ag</literal></entry>
+ <entry>
+ Tokens that appears as phrases (with blank in them) gets
+ structure phrase attached (4=1). Tokens that appear to be words
+ gets structure word attached (4=2). Phrases and words are
+ ANDed. This is a variant of s=al and s=pw, with the main
+ difference that words are not split (with operator AND)
+ but instead kept in one RPN token. This facility appeared
+ in YAZ 4.2.38.
+ </entry>
+ </row>
+ <row><entry><literal>r=o</literal></entry>
+ <entry>
+ Allows ranges and the operators greather-than, less-than, ...
+ equals.
+ This sets Bib-1 relation attribute accordingly (relation
+ ordered). A query construct is only treated as a range if
+ dash is used and that is surrounded by white-space. So
+ <literal>-1980</literal> is treated as term
+ <literal>"-1980"</literal> not <literal><= 1980</literal>.
+ If <literal>- 1980</literal> is used, however, that is
+ treated as a range.
+ </entry>
+ </row>
+ <row><entry><literal>r=r</literal></entry>
+ <entry>
+ Similar to <literal>r=o</literal> but assumes that terms
+ are non-negative (not prefixed with <literal>-</literal>).
+ Thus, a dash will always be treated as a range.
+ The construct <literal>1980-1990</literal> is
+ treated as a range with <literal>r=r</literal> but as a
+ single term <literal>"1980-1990"</literal> with
+ <literal>r=o</literal>. The special attribute
+ <literal>r=r</literal> is available in YAZ 2.0.24 or later.
+ </entry>
+ </row>
+ <row><entry><literal>t=l</literal></entry>
+ <entry>
+ Allows term to be left-truncated.
+ If term is of the form <literal>?x</literal>, the resulting
+ Type-1 term is <literal>x</literal> and truncation is left.
+ </entry>
+ </row>
+ <row><entry><literal>t=r</literal></entry>
+ <entry>
+ Allows term to be right-truncated.
+ If term is of the form <literal>x?</literal>, the resulting
+ Type-1 term is <literal>x</literal> and truncation is right.
+ </entry>
+ </row>
+ <row><entry><literal>t=n</literal></entry>
+ <entry>
+ If term is does not include <literal>?</literal>, the
+ truncation attribute is set to none (100).
+ </entry>
+ </row>
+ <row><entry><literal>t=b</literal></entry>
+ <entry>
+ Allows term to be both left&right truncated.
+ If term is of the form <literal>?x?</literal>, the
+ resulting term is <literal>x</literal> and trunctation is
+ set to both left&right.
+ </entry>
+ </row>
+ <row><entry><literal>t=x</literal></entry>
+ <entry>
+ Allows masking anywhere in a term, thus fully supporting
+ # (mask one character) and ? (zero or more of any).
+ If masking is used, trunction is set to 102 (regexp-1 in term)
+ and the term is converted accordingly to a regular expression.
+ </entry>
+ </row>
+ <row><entry><literal>t=z</literal></entry>
+ <entry>
+ Allows masking anywhere in a term, thus fully supporting
+ # (mask one character) and ? (zero or more of any).
+ If masking is used, trunction is set to 104 (Z39.58 in term)
+ and the term is converted accordingly to Z39.58 masking term -
+ actually the same truncation as CCL itself.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </para>
+ <example id="example.ccl.profile">
+ <title>CCL profile</title>
+ <para>
+ Consider the following definition:
+ </para>
+ <screen>
+ ti u=4 s=1
+ au u=1 s=1
+ term s=105
+ ranked r=102
+ date u=30 r=o
+ </screen>
+ <para>
+ <literal>ti</literal> and <literal>au</literal> both set
+ structure attribute to phrase (s=1).
+ <literal>ti</literal>
+ sets the use-attribute to 4. <literal>au</literal> sets the
+ use-attribute to 1.
+ When no qualifiers are used in the query the structure-attribute is
+ set to free-form-text (105) (rule for <literal>term</literal>).
+ The <literal>date</literal> sets the relation attribute to
+ the relation used in the CCL query and sets the use attribute
+ to 30 (Bib-1 Date).
+ </para>
+ <para>
+ You can combine attributes. To Search for "ranked title" you
+ can do
+ <screen>
+ ti,ranked=knuth computer
+ </screen>
+ which will set relation=ranked, use=title, structure=phrase.
+ </para>
+ <para>
+ Query
+ <screen>
+ date > 1980
+ </screen>
+ is a valid query. But
+ <screen>
+ ti > 1980
+ </screen>
+ is invalid.
+ </para>
+ </example>
+ </sect4>
+ <sect4 id="ccl.qualifier.alias">
+ <title>Qualifier alias</title>
+ <para>
+ A qualifier alias is of the form:
+ </para>
+ <para>
+ <replaceable>q</replaceable>
+ <replaceable>q1</replaceable> <replaceable>q2</replaceable> ..
+ </para>
+ <para>
+ which declares <replaceable>q</replaceable> to
+ be an alias for <replaceable>q1</replaceable>,
+ <replaceable>q2</replaceable>... such that the CCL
+ query <replaceable>q=x</replaceable> is equivalent to
+ <replaceable>q1=x or q2=x or ...</replaceable>.
+ </para>
+ </sect4>
+ <sect4 id="ccl.comments">
+ <title>Comments</title>
+ <para>
+ Lines with white space or lines that begin with
+ character <literal>#</literal> are treated as comments.
+ </para>
+ </sect4>
+ <sect4 id="ccl.directives">
+ <title>Directives</title>
+ <para>
+ Directive specifications takes the form
+ </para>
+ <para><literal>@</literal><replaceable>directive</replaceable> <replaceable>value</replaceable>
+ </para>
+ <table id="ccl.directives.table">
+ <title>CCL directives</title>
+ <tgroup cols="3">
+ <colspec colwidth="2*" colname="name"></colspec>
+ <colspec colwidth="8*" colname="description"></colspec>
+ <colspec colwidth="1*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>truncation</entry>
+ <entry>Truncation character</entry>
+ <entry><literal>?</literal></entry>
+ </row>
+ <row>
+ <entry>mask</entry>
+ <entry>Masking character. Requires YAZ 4.2.58 or later</entry>
+ <entry><literal>#</literal></entry>
+ </row>
+ <row>
+ <entry>field</entry>
+ <entry>Specifies how multiple fields are to be
+ combined. There are two modes: <literal>or</literal>:
+ multiple qualifier fields are ORed,
+ <literal>merge</literal>: attributes for the qualifier
+ fields are merged and assigned to one term.
+ </entry>
+ <entry><literal>merge</literal></entry>
+ </row>
+ <row>
+ <entry>case</entry>
+ <entry>Specifies if CCL operators and qualifiers should be
+ compared with case sensitivity or not. Specify 1 for
+ case sensitive; 0 for case insensitive.</entry>
+ <entry><literal>1</literal></entry>
+ </row>
+ <row>
+ <entry>and</entry>
+ <entry>Specifies token for CCL operator AND.</entry>
+ <entry><literal>and</literal></entry>
+ </row>
+ <row>
+ <entry>or</entry>
+ <entry>Specifies token for CCL operator OR.</entry>
+ <entry><literal>or</literal></entry>
+ </row>
+ <row>
+ <entry>not</entry>
+ <entry>Specifies token for CCL operator NOT.</entry>
+ <entry><literal>not</literal></entry>
+ </row>
+ <row>
+ <entry>set</entry>
+ <entry>Specifies token for CCL operator SET.</entry>
+ <entry><literal>set</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect4>
+ </sect3>
+ <sect3 id="ccl.api">
+ <title>CCL API</title>
+ <para>
+ All public definitions can be found in the header file
+ <filename>ccl.h</filename>. A profile identifier is of type
+ <literal>CCL_bibset</literal>. A profile must be created with the call
+ to the function <function>ccl_qual_mk</function> which returns a profile
+ handle of type <literal>CCL_bibset</literal>.
+ </para>
+ <para>
+ To read a file containing qualifier definitions the function
+ <function>ccl_qual_file</function> may be convenient. This function
+ takes an already opened <literal>FILE</literal> handle pointer as
+ argument along with a <literal>CCL_bibset</literal> handle.
+ </para>
+ <para>
+ To parse a simple string with a FIND query use the function
+ </para>
+ <screen>
+struct ccl_rpn_node *ccl_find_str(CCL_bibset bibset, const char *str,
+ int *error, int *pos);
+ </screen>
+ <para>
+ which takes the CCL profile (<literal>bibset</literal>) and query
+ (<literal>str</literal>) as input. Upon successful completion the RPN
+ tree is returned. If an error occur, such as a syntax error, the integer
+ pointed to by <literal>error</literal> holds the error code and
+ <literal>pos</literal> holds the offset inside query string in which
+ the parsing failed.
+ </para>
+ <para>
+ An English representation of the error may be obtained by calling
+ the <literal>ccl_err_msg</literal> function. The error codes are
+ listed in <filename>ccl.h</filename>.
+ </para>
+ <para>
+ To convert the CCL RPN tree (type
+ <literal>struct ccl_rpn_node *</literal>)
+ to the Z_RPNQuery of YAZ the function <function>ccl_rpn_query</function>
+ must be used. This function which is part of YAZ is implemented in
+ <filename>yaz-ccl.c</filename>.
+ After calling this function the CCL RPN tree is probably no longer
+ needed. The <literal>ccl_rpn_delete</literal> destroys the CCL RPN tree.
+ </para>
+ <para>
+ A CCL profile may be destroyed by calling the
+ <function>ccl_qual_rm</function> function.
+ </para>
+ <para>
+ The token names for the CCL operators may be changed by setting the
+ globals (all type <literal>char *</literal>)
+ <literal>ccl_token_and</literal>, <literal>ccl_token_or</literal>,
+ <literal>ccl_token_not</literal> and <literal>ccl_token_set</literal>.
+ An operator may have aliases, i.e. there may be more than one name for
+ the operator. To do this, separate each alias with a space character.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2 id="cql">
+ <title>CQL</title>
+ <para>
+ <ulink url="&url.cql;">CQL</ulink>
+ - Common Query Language - was defined for the
+ <ulink url="&url.sru;">SRU</ulink> protocol.
+ In many ways CQL has a similar syntax to CCL.
+ The objective of CQL is different. Where CCL aims to be
+ an end-user language, CQL is <emphasis>the</emphasis> protocol
+ query language for SRU.
+ </para>
+ <tip>
+ <para>
+ If you are new to CQL, read the
+ <ulink url="&url.cql.intro;">Gentle Introduction</ulink>.
+ </para>
+ </tip>
+ <para>
+ The CQL parser in &yaz; provides the following:
+ <itemizedlist>
+ <listitem>
+ <para>
+ It parses and validates a CQL query.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ It generates a C structure that allows you to convert
+ a CQL query to some other query language, such as SQL.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The parser converts a valid CQL query to PQF, thus providing a
+ way to use CQL for both SRU servers and Z39.50 targets at the
+ same time.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The parser converts CQL to XCQL.
+ XCQL is an XML representation of CQL.
+ XCQL is part of the SRU specification. However, since SRU
+ supports CQL only, we don't expect XCQL to be widely used.
+ Furthermore, CQL has the advantage over XCQL that it is
+ easy to read.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <sect3 id="cql.parsing">
+ <title>CQL parsing</title>
+ <para>
+ A CQL parser is represented by the <literal>CQL_parser</literal>
+ handle. Its contents should be considered &yaz; internal (private).
+ <synopsis>
+#include <yaz/cql.h>
+
+typedef struct cql_parser *CQL_parser;
+
+CQL_parser cql_parser_create(void);
+void cql_parser_destroy(CQL_parser cp);
+ </synopsis>
+ A parser is created by <function>cql_parser_create</function> and
+ is destroyed by <function>cql_parser_destroy</function>.
+ </para>
+ <para>
+ To parse a CQL query string, the following function
+ is provided:
+ <synopsis>
+int cql_parser_string(CQL_parser cp, const char *str);
+ </synopsis>
+ A CQL query is parsed by the <function>cql_parser_string</function>
+ which takes a query <parameter>str</parameter>.
+ If the query was valid (no syntax errors), then zero is returned;
+ otherwise -1 is returned to indicate a syntax error.
+ </para>
+ <para>
+ <synopsis>
+int cql_parser_stream(CQL_parser cp,
+ int (*getbyte)(void *client_data),
+ void (*ungetbyte)(int b, void *client_data),
+ void *client_data);
+
+int cql_parser_stdio(CQL_parser cp, FILE *f);
+ </synopsis>
+ The functions <function>cql_parser_stream</function> and
+ <function>cql_parser_stdio</function> parses a CQL query
+ - just like <function>cql_parser_string</function>.
+ The only difference is that the CQL query can be
+ fed to the parser in different ways.
+ The <function>cql_parser_stream</function> uses a generic
+ byte stream as input. The <function>cql_parser_stdio</function>
+ uses a <literal>FILE</literal> handle which is opened for reading.
+ </para>
+ </sect3>
+ <sect3 id="cql.tree">
+ <title>CQL tree</title>
+ <para>
+ The the query string is valid, the CQL parser
+ generates a tree representing the structure of the
+ CQL query.
+ </para>
+ <para>
+ <synopsis>
+struct cql_node *cql_parser_result(CQL_parser cp);
+ </synopsis>
+ <function>cql_parser_result</function> returns the
+ a pointer to the root node of the resulting tree.
+ </para>
+ <para>
+ Each node in a CQL tree is represented by a
+ <literal>struct cql_node</literal>.
+ It is defined as follows:
+ <synopsis>
+#define CQL_NODE_ST 1
+#define CQL_NODE_BOOL 2
+#define CQL_NODE_SORT 3
+struct cql_node {
+ int which;
+ union {
+ struct {
+ char *index;
+ char *index_uri;
+ char *term;
+ char *relation;
+ char *relation_uri;
+ struct cql_node *modifiers;
+ } st;
+ struct {
+ char *value;
+ struct cql_node *left;
+ struct cql_node *right;
+ struct cql_node *modifiers;
+ } boolean;
+ struct {
+ char *index;
+ struct cql_node *next;
+ struct cql_node *modifiers;
+ struct cql_node *search;
+ } sort;
+ } u;
+};
+ </synopsis>
+ There are three node types: search term (ST), boolean (BOOL)
+ and sortby (SORT).
+ A modifier is treated as a search term too.
+ </para>
+ <para>
+ The search term node has five members:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>index</literal>: index for search term.
+ If an index is unspecified for a search term,
+ <literal>index</literal> will be NULL.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>index_uri</literal>: index URi for search term
+ or NULL if none could be resolved for the index.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>term</literal>: the search term itself.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>relation</literal>: relation for search term.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>relation_uri</literal>: relation URI for search term.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>modifiers</literal>: relation modifiers for search
+ term. The <literal>modifiers</literal> list itself of cql_nodes
+ each of type <literal>ST</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The boolean node represents <literal>and</literal>,
+ <literal>or</literal>, <literal>not</literal> +
+ proximity.
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>left</literal> and <literal>right</literal>: left
+ - and right operand respectively.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>modifiers</literal>: proximity arguments.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The sort node represents both the SORTBY clause.
+ </para>
+ </sect3>
+ <sect3 id="cql.to.pqf">
+ <title>CQL to PQF conversion</title>
+ <para>
+ Conversion to PQF (and Z39.50 RPN) is tricky by the fact
+ that the resulting RPN depends on the Z39.50 target
+ capabilities (combinations of supported attributes).
+ In addition, the CQL and SRU operates on index prefixes
+ (URI or strings), whereas the RPN uses Object Identifiers
+ for attribute sets.
+ </para>
+ <para>
+ The CQL library of &yaz; defines a <literal>cql_transform_t</literal>
+ type. It represents a particular mapping between CQL and RPN.
+ This handle is created and destroyed by the functions:
+ <synopsis>
+cql_transform_t cql_transform_open_FILE (FILE *f);
+cql_transform_t cql_transform_open_fname(const char *fname);
+void cql_transform_close(cql_transform_t ct);
+ </synopsis>
+ The first two functions create a tranformation handle from
+ either an already open FILE or from a filename respectively.
+ </para>
+ <para>
+ The handle is destroyed by <function>cql_transform_close</function>
+ in which case no further reference of the handle is allowed.
+ </para>
+ <para>
+ When a <literal>cql_transform_t</literal> handle has been created
+ you can convert to RPN.
+ <synopsis>
+int cql_transform_buf(cql_transform_t ct,
+ struct cql_node *cn, char *out, int max);
+ </synopsis>
+ This function converts the CQL tree <literal>cn</literal>
+ using handle <literal>ct</literal>.
+ For the resulting PQF, you supply a buffer <literal>out</literal>
+ which must be able to hold at at least <literal>max</literal>
+ characters.
+ </para>
+ <para>
+ If conversion failed, <function>cql_transform_buf</function>
+ returns a non-zero SRU error code; otherwise zero is returned
+ (conversion successful). The meanings of the numeric error
+ codes are listed in the SRU specification somewhere (no
+ direct link anymore).
+ </para>
+ <para>
+ If conversion fails, more information can be obtained by calling
+ <synopsis>
+int cql_transform_error(cql_transform_t ct, char **addinfop);
+ </synopsis>
+ This function returns the most recently returned numeric
+ error-code and sets the string-pointer at
+ <literal>*addinfop</literal> to point to a string containing
+ additional information about the error that occurred: for
+ example, if the error code is 15 (``Illegal or unsupported context
+ set''), the additional information is the name of the requested
+ context set that was not recognised.
+ </para>
+ <para>
+ The SRU error-codes may be translated into brief human-readable
+ error messages using
+ <synopsis>
+const char *cql_strerror(int code);
+ </synopsis>
+ </para>
+ <para>
+ If you wish to be able to produce a PQF result in a different
+ way, there are two alternatives.
+ <synopsis>
+void cql_transform_pr(cql_transform_t ct,
+ struct cql_node *cn,
+ void (*pr)(const char *buf, void *client_data),
+ void *client_data);
+
+int cql_transform_FILE(cql_transform_t ct,
+ struct cql_node *cn, FILE *f);
+ </synopsis>
+ The former function produces output to a user-defined
+ output stream. The latter writes the result to an already
+ open <literal>FILE</literal>.
+ </para>
+ </sect3>
+ <sect3 id="cql.to.rpn">
+ <title>Specification of CQL to RPN mappings</title>
+ <para>
+ The file supplied to functions
+ <function>cql_transform_open_FILE</function>,
+ <function>cql_transform_open_fname</function> follows
+ a structure found in many Unix utilities.
+ It consists of mapping specifications - one per line.
+ Lines starting with <literal>#</literal> are ignored (comments).
+ </para>
+ <para>
+ Each line is of the form
+ <literallayout>
+ <replaceable>CQL pattern</replaceable><literal> = </literal> <replaceable> RPN equivalent</replaceable>
+ </literallayout>
+ </para>
+ <para>
+ An RPN pattern is a simple attribute list. Each attribute pair
+ takes the form:
+ <literallayout>
+ [<replaceable>set</replaceable>] <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>
+ </literallayout>
+ The attribute <replaceable>set</replaceable> is optional.
+ The <replaceable>type</replaceable> is the attribute type,
+ <replaceable>value</replaceable> the attribute value.
+ </para>
+ <para>
+ The character <literal>*</literal> (asterisk) has special meaning
+ when used in the RPN pattern.
+ Each occurrence of <literal>*</literal> is substituted with the
+ CQL matching name (index, relation, qualifier etc).
+ This facility can be used to copy a CQL name verbatim to the RPN result.
+ </para>
+ <para>
+ The following CQL patterns are recognized:
+ <variablelist>
+ <varlistentry>
+ <term>
+ <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern is invoked when a CQL index, such as
+ dc.title is converted. <replaceable>set</replaceable>
+ and <replaceable>name</replaceable> are the context set and index
+ name respectively.
+ Typically, the RPN specifies an equivalent use attribute.
+ </para>
+ <para>
+ For terms not bound by an index the pattern
+ <literal>index.cql.serverChoice</literal> is used.
+ Here, the prefix <literal>cql</literal> is defined as
+ <literal>http://www.loc.gov/zing/cql/cql-indexes/v1.0/</literal>.
+ If this pattern is not defined, the mapping will fail.
+ </para>
+ <para>
+ The pattern,
+ <literal>index.</literal><replaceable>set</replaceable><literal>.*</literal>
+ is used when no other index pattern is matched.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>qualifier.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
+ (DEPRECATED)
+ </term>
+ <listitem>
+ <para>
+ For backwards compatibility, this is recognised as a synonym of
+ <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>relation.</literal><replaceable>relation</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how a CQL relation is mapped to RPN.
+ <replaceable>pattern</replaceable> is name of relation
+ operator. Since <literal>=</literal> is used as
+ separator between CQL pattern and RPN, CQL relations
+ including <literal>=</literal> cannot be
+ used directly. To avoid a conflict, the names
+ <literal>ge</literal>,
+ <literal>eq</literal>,
+ <literal>le</literal>,
+ must be used for CQL operators, greater-than-or-equal,
+ equal, less-than-or-equal respectively.
+ The RPN pattern is supposed to include a relation attribute.
+ </para>
+ <para>
+ For terms not bound by a relation, the pattern
+ <literal>relation.scr</literal> is used. If the pattern
+ is not defined, the mapping will fail.
+ </para>
+ <para>
+ The special pattern, <literal>relation.*</literal> is used
+ when no other relation pattern is matched.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>relationModifier.</literal><replaceable>mod</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how a CQL relation modifier is mapped to RPN.
+ The RPN pattern is usually a relation attribute.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>structure.</literal><replaceable>type</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how a CQL structure is mapped to RPN.
+ Note that this CQL pattern is somewhat to similar to
+ CQL pattern <literal>relation</literal>.
+ The <replaceable>type</replaceable> is a CQL relation.
+ </para>
+ <para>
+ The pattern, <literal>structure.*</literal> is used
+ when no other structure pattern is matched.
+ Usually, the RPN equivalent specifies a structure attribute.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>position.</literal><replaceable>type</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how the anchor (position) of
+ CQL is mapped to RPN.
+ The <replaceable>type</replaceable> is one
+ of <literal>first</literal>, <literal>any</literal>,
+ <literal>last</literal>, <literal>firstAndLast</literal>.
+ </para>
+ <para>
+ The pattern, <literal>position.*</literal> is used
+ when no other position pattern is matched.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>set.</literal><replaceable>prefix</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This specification defines a CQL context set for a given prefix.
+ The value on the right hand side is the URI for the set -
+ <emphasis>not</emphasis> RPN. All prefixes used in
+ index patterns must be defined this way.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>set</literal>
+ </term>
+ <listitem>
+ <para>
+ This specification defines a default CQL context set for index names.
+ The value on the right hand side is the URI for the set.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <example id="example.cql.to.rpn.mapping">
+ <title>CQL to RPN mapping file</title>
+ <para>
+ This simple file defines two context sets, three indexes and three
+ relations, a position pattern and a default structure.
+ </para>
+ <programlisting><![CDATA[
+ set.cql = http://www.loc.gov/zing/cql/context-sets/cql/v1.1/
+ set.dc = http://www.loc.gov/zing/cql/dc-indexes/v1.0/
+
+ index.cql.serverChoice = 1=1016
+ index.dc.title = 1=4
+ index.dc.subject = 1=21
+
+ relation.< = 2=1
+ relation.eq = 2=3
+ relation.scr = 2=3
+
+ position.any = 3=3 6=1
+
+ structure.* = 4=1
+]]>
+ </programlisting>
+ <para>
+ With the mappings above, the CQL query
+ <screen>
+ computer
+ </screen>
+ is converted to the PQF:
+ <screen>
+ @attr 1=1016 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "computer"
+ </screen>
+ by rules <literal>index.cql.serverChoice</literal>,
+ <literal>relation.scr</literal>, <literal>structure.*</literal>,
+ <literal>position.any</literal>.
+ </para>
+ <para>
+ CQL query
+ <screen>
+ computer^
+ </screen>
+ is rejected, since <literal>position.right</literal> is
+ undefined.
+ </para>
+ <para>
+ CQL query
+ <screen>
+ >my = "http://www.loc.gov/zing/cql/dc-indexes/v1.0/" my.title = x
+ </screen>
+ is converted to
+ <screen>
+ @attr 1=4 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "x"
+ </screen>
+ </para>
+ </example>
+ <example id="example.cql.to.rpn.string">
+ <title>CQL to RPN string attributes</title>
+ <para>
+ In this example we allow any index to be passed to RPN as
+ a use attribute.
+ </para>
+ <programlisting><![CDATA[
+ # Identifiers for prefixes used in this file. (index.*)
+ set.cql = info:srw/cql-context-set/1/cql-v1.1
+ set.rpn = http://bogus/rpn
+ set = http://bogus/rpn
+
+ # The default index when none is specified by the query
+ index.cql.serverChoice = 1=any
+
+ index.rpn.* = 1=*
+ relation.eq = 2=3
+ structure.* = 4=1
+ position.any = 3=3
+]]>
+ </programlisting>
+ <para>
+ The <literal>http://bogus/rpn</literal> context set is also the default
+ so we can make queries such as
+ <screen>
+ title = a
+ </screen>
+ which is converted to
+ <screen>
+ @attr 2=3 @attr 4=1 @attr 3=3 @attr 1=title "a"
+ </screen>
+ </para>
+ </example>
+ <example id="example.cql.to.rpn.bathprofile">
+ <title>CQL to RPN using Bath Profile</title>
+ <para>
+ The file <filename>etc/pqf.properties</filename> has mappings from
+ the Bath Profile and Dublin Core to RPN.
+ If YAZ is installed as a package it's usually located
+ in <filename>/usr/share/yaz/etc</filename> and part of the
+ development package, such as <literal>libyaz-dev</literal>.
+ </para>
+ </example>
+ </sect3>
+ <sect3 id="cql.xcql">
+ <title>CQL to XCQL conversion</title>
+ <para>
+ Conversion from CQL to XCQL is trivial and does not
+ require a mapping to be defined.
+ There three functions to choose from depending on the
+ way you wish to store the resulting output (XML buffer
+ containing XCQL).
+ <synopsis>
+int cql_to_xml_buf(struct cql_node *cn, char *out, int max);
+void cql_to_xml(struct cql_node *cn,
+ void (*pr)(const char *buf, void *client_data),
+ void *client_data);
+void cql_to_xml_stdio(struct cql_node *cn, FILE *f);
+ </synopsis>
+ Function <function>cql_to_xml_buf</function> converts
+ to XCQL and stores result in a user supplied buffer of a given
+ max size.
+ </para>
+ <para>
+ <function>cql_to_xml</function> writes the result in
+ a user defined output stream.
+ <function>cql_to_xml_stdio</function> writes to a
+ a file.
+ </para>
+ </sect3>
+ <sect3 id="rpn.to.cql">
+ <title>PQF to CQL conversion</title>
+ <para>
+ Conversion from PQF to CQL is offered by the two functions shown
+ below. The former uses a generic stream for result. The latter
+ puts result in a WRBUF (string container).
+ <synopsis>
+#include <yaz/rpn2cql.h>
+
+int cql_transform_rpn2cql_stream(cql_transform_t ct,
+ void (*pr)(const char *buf, void *client_data),
+ void *client_data,
+ Z_RPNQuery *q);
+
+int cql_transform_rpn2cql_wrbuf(cql_transform_t ct,
+ WRBUF w,
+ Z_RPNQuery *q);
+ </synopsis>
+ The configuration is the same as used in CQL to PQF conversions.
+ </para>
+ </sect3>
+ </sect2>
+ </sect1>
+ <sect1 id="tools.oid">
+ <title>Object Identifiers</title>
+ <para>
+ The basic YAZ representation of an OID is an array of integers,
+ terminated with the value -1. This integer is of type
+ <literal>Odr_oid</literal>.
+ </para>
+ <para>
+ Fundamental OID operations and the type <literal>Odr_oid</literal>
+ are defined in <filename>yaz/oid_util.h</filename>.
+ </para>
+ <para>
+ An OID can either be declared as a automatic variable or it can
+ allocated using the memory utilities or ODR/NMEM. It's
+ guaranteed that an OID can fit in <literal>OID_SIZE</literal> integers.
+ </para>
+ <example id="tools.oid.bib1.1"><title>Create OID on stack</title>
+ <para>
+ We can create an OID for the Bib-1 attribute set with:
+ <screen>
+ Odr_oid bib1[OID_SIZE];
+ bib1[0] = 1;
+ bib1[1] = 2;
+ bib1[2] = 840;
+ bib1[3] = 10003;
+ bib1[4] = 3;
+ bib1[5] = 1;
+ bib1[6] = -1;
+ </screen>
+ </para>
+ </example>
+ <para>
+ And OID may also be filled from a string-based representation using
+ dots (.). This is achieved by function
+ <screen>
+ int oid_dotstring_to_oid(const char *name, Odr_oid *oid);
+ </screen>
+ This functions returns 0 if name could be converted; -1 otherwise.
+ </para>
+ <example id="tools.oid.bib1.2"><title>Using oid_oiddotstring_to_oid</title>
+ <para>
+ We can fill the Bib-1 attribute set OID easier with:
+ <screen>
+ Odr_oid bib1[OID_SIZE];
+ oid_oiddotstring_to_oid("1.2.840.10003.3.1", bib1);
+ </screen>
+ </para>
+ </example>
+ <para>
+ We can also allocate an OID dynamically on a ODR stream with:
+ <screen>
+ Odr_oid *odr_getoidbystr(ODR o, const char *str);
+ </screen>
+ This creates an OID from string-based representation using dots.
+ This function take an &odr; stream as parameter. This stream is used to
+ allocate memory for the data elements, which is released on a
+ subsequent call to <function>odr_reset()</function> on that stream.
+ </para>
+ <example id="tools.oid.bib1.3">
+ <title>Using odr_getoidbystr</title>
+ <para>
+ We can create a OID for the Bib-1 attribute set with:
+ <screen>
+ Odr_oid *bib1 = odr_getoidbystr(odr, "1.2.840.10003.3.1");
+ </screen>
+ </para>
+ </example>
+ <para>
+ The function
+ <screen>
+ char *oid_oid_to_dotstring(const Odr_oid *oid, char *oidbuf)
+ </screen>
+ does the reverse of <function>oid_oiddotstring_to_oid</function>. It
+ converts an OID to the string-based representation using dots.
+ The supplied char buffer <literal>oidbuf</literal> holds the resulting
+ string and must be at least <literal>OID_STR_MAX</literal> in size.
+ </para>
+ <para>
+ OIDs can be copied with <function>oid_oidcpy</function> which takes
+ two OID lists as arguments. Alternativly, an OID copy can be allocated
+ on a ODR stream with:
+ <screen>
+ Odr_oid *odr_oiddup(ODR odr, const Odr_oid *o);
+ </screen>
+ </para>
+ <para>
+ OIDs can be compared with <function>oid_oidcmp</function> which returns
+ zero if the two OIDs provided are identical; non-zero otherwise.
+ </para>
+ <sect2 id="tools.oid.database">
+ <title>OID database</title>
+ <para>
+ From YAZ version 3 and later, the oident system has been replaced
+ by an OID database. OID database is a misnomer .. the old odient
+ system was also a database.
+ </para>
+ <para>
+ The OID database is really just a map between named Object Identifiers
+ (string) and their OID raw equivalents. Most operations either
+ convert from string to OID or other way around.
+ </para>
+ <para>
+ Unfortunately, whenever we supply a string we must also specify the
+ <emphasis>OID class</emphasis>. The class is necessary because some
+ strings correspond to multiple OIDs. An example of such a string is
+ <literal>Bib-1</literal> which may either be an attribute-set
+ or a diagnostic-set.
+ </para>
+ <para>
+ Applications using the YAZ database should include
+ <filename>yaz/oid_db.h</filename>.
+ </para>
+ <para>
+ A YAZ database handle is of type <literal>yaz_oid_db_t</literal>.
+ Actually that's a pointer. You need not think deal with that.
+ YAZ has a built-in database which can be considered "constant" for
+ most purposes.
+ We can get hold that by using function <function>yaz_oid_std</function>.
+ </para>
+ <para>
+ All functions with prefix <function>yaz_string_to_oid</function>
+ converts from class + string to OID. We have variants of this
+ operation due to different memory allocation strategies.
+ </para>
+ <para>
+ All functions with prefix
+ <function>yaz_oid_to_string</function> converts from OID to string
+ + class.
+ </para>
+ <example id="tools.oid.bib1.4">
+ <title>Create OID with YAZ DB</title>
+ <para>
+ We can create an OID for the Bib-1 attribute set on the ODR stream
+ odr with:
+ <screen>
+ Odr_oid *bib1 =
+ yaz_string_to_oid_odr(yaz_oid_std(), CLASS_ATTSET, "Bib-1", odr);
+ </screen>
+ This is more complex than using <function>odr_getoidbystr</function>.
+ You would only use <function>yaz_string_to_oid_odr</function> when the
+ string (here Bib-1) is supplied by a user or configuration.
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="tools.oid.std">
+ <title>Standard OIDs</title>
+ <para>
+ All the object identifers in the standard OID database as returned
+ by <function>yaz_oid_std</function> can referenced directly in a
+ program as a constant OID.
+ Each constant OID is prefixed with <literal>yaz_oid_</literal> -
+ followed by OID class (lowercase) - then by OID name (normalized and
+ lowercase).
+ </para>
+ <para>
+ See <xref linkend="list-oids"/> for list of all object identifiers
+ built into YAZ.
+ These are declared in <filename>yaz/oid_std.h</filename> but are
+ included by <filename>yaz/oid_db.h</filename> as well.
+ </para>
+ <example id="tools.oid.bib1.5">
+ <title>Use a built-in OID</title>
+ <para>
+ We can allocate our own OID filled with the constant OID for
+ Bib-1 with:
+ <screen>
+ Odr_oid *bib1 = odr_oiddup(o, yaz_oid_attset_bib1);
+ </screen>
+ </para>
+ </example>
+ </sect2>
+ </sect1>
+ <sect1 id="tools.nmem">
+ <title>Nibble Memory</title>
+ <para>
+ Sometimes when you need to allocate and construct a large,
+ interconnected complex of structures, it can be a bit of a pain to
+ release the associated memory again. For the structures describing the
+ Z39.50 PDUs and related structures, it is convenient to use the
+ memory-management system of the &odr; subsystem (see
+ <xref linkend="odr.use"/>). However, in some circumstances
+ where you might otherwise benefit from using a simple nibble memory
+ management system, it may be impractical to use
+ <function>odr_malloc()</function> and <function>odr_reset()</function>.
+ For this purpose, the memory manager which also supports the &odr;
+ streams is made available in the NMEM module. The external interface
+ to this module is given in the <filename>nmem.h</filename> file.
+ </para>
+ <para>
+ The following prototypes are given:
+ </para>
+ <screen>
+ NMEM nmem_create(void);
+ void nmem_destroy(NMEM n);
+ void *nmem_malloc(NMEM n, size_t size);
+ void nmem_reset(NMEM n);
+ size_t nmem_total(NMEM n);
+ void nmem_init(void);
+ void nmem_exit(void);
+ </screen>
+ <para>
+ The <function>nmem_create()</function> function returns a pointer to a
+ memory control handle, which can be released again by
+ <function>nmem_destroy()</function> when no longer needed.
+ The function <function>nmem_malloc()</function> allocates a block of
+ memory of the requested size. A call to <function>nmem_reset()</function>
+ or <function>nmem_destroy()</function> will release all memory allocated
+ on the handle since it was created (or since the last call to
+ <function>nmem_reset()</function>. The function
+ <function>nmem_total()</function> returns the number of bytes currently
+ allocated on the handle.
+ </para>
+ <para>
+ The nibble memory pool is shared amongst threads. POSIX
+ mutex'es and WIN32 Critical sections are introduced to keep the
+ module thread safe. Function <function>nmem_init()</function>
+ initializes the nibble memory library and it is called automatically
+ the first time the <literal>YAZ.DLL</literal> is loaded. &yaz; uses
+ function <function>DllMain</function> to achieve this. You should
+ <emphasis>not</emphasis> call <function>nmem_init</function> or
+ <function>nmem_exit</function> unless you're absolute sure what
+ you're doing. Note that in previous &yaz; versions you'd have to call
+ <function>nmem_init</function> yourself.
+ </para>
+ </sect1>
+ <sect1 id="tools.log">
+ <title>Log</title>
+ <para>
+ &yaz; has evolved a fairly complex log system which should be useful both
+ for debugging &yaz; itself, debugging applications that use &yaz;, and for
+ production use of those applications.
+ </para>
+ <para>
+ The log functions are declared in header <filename>yaz/log.h</filename>
+ and implemented in <filename>src/log.c</filename>.
+ Due to name clash with syslog and some math utilities the logging
+ interface has been modified as of YAZ 2.0.29. The obsolete interface
+ is still available if in header file <filename>yaz/log.h</filename>.
+ The key points of the interface are:
+ </para>
+ <screen>
+ void yaz_log(int level, const char *fmt, ...)
+ void yaz_log_init(int level, const char *prefix, const char *name);
+ void yaz_log_init_file(const char *fname);
+ void yaz_log_init_level(int level);
+ void yaz_log_init_prefix(const char *prefix);
+ void yaz_log_time_format(const char *fmt);
+ void yaz_log_init_max_size(int mx);
+
+ int yaz_log_mask_str(const char *str);
+ int yaz_log_module_level(const char *name);
+ </screen>
+ <para>
+ The reason for the whole log module is the <function>yaz_log</function>
+ function. It takes a bitmask indicating the log levels, a
+ <literal>printf</literal>-like format string, and a variable number of
+ arguments to log.
+ </para>
+ <para>
+ The <literal>log level</literal> is a bit mask, that says on which level(s)
+ the log entry should be made, and optionally set some behaviour of the
+ logging. In the most simple cases, it can be one of <literal>YLOG_FATAL,
+ YLOG_DEBUG, YLOG_WARN, YLOG_LOG</literal>. Those can be combined with bits
+ that modify the way the log entry is written:<literal>YLOG_ERRNO,
+ YLOG_NOTIME, YLOG_FLUSH</literal>.
+ Most of the rest of the bits are deprecated, and should not be used. Use
+ the dynamic log levels instead.
+ </para>
+ <para>
+ Applications that use &yaz;, should not use the LOG_LOG for ordinary
+ messages, but should make use of the dynamic loglevel system. This consists
+ of two parts, defining the loglevel and checking it.
+ </para>
+ <para>
+ To define the log levels, the (main) program should pass a string to
+ <function>yaz_log_mask_str</function> to define which log levels are to be
+ logged. This string should be a comma-separated list of log level names,
+ and can contain both hard-coded names and dynamic ones. The log level
+ calculation starts with <literal>YLOG_DEFAULT_LEVEL</literal> and adds a bit
+ for each word it meets, unless the word starts with a '-', in which case it
+ clears the bit. If the string <literal>'none'</literal> is found,
+ all bits are cleared. Typically this string comes from the command-line,
+ often identified by <literal>-v</literal>. The
+ <function>yaz_log_mask_str</function> returns a log level that should be
+ passed to <function>yaz_log_init_level</function> for it to take effect.
+ </para>
+ <para>
+ Each module should check what log bits it should be used, by calling
+ <function>yaz_log_module_level</function> with a suitable name for the
+ module. The name is cleared from a preceding path and an extension, if any,
+ so it is quite possible to use <literal>__FILE__</literal> for it. If the
+ name has been passed to <function>yaz_log_mask_str</function>, the routine
+ returns a non-zero bitmask, which should then be used in consequent calls
+ to yaz_log. (It can also be tested, so as to avoid unnecessary calls to
+ yaz_log, in time-critical places, or when the log entry would take time
+ to construct.)
+ </para>
+ <para>
+ Yaz uses the following dynamic log levels:
+ <literal>server, session, request, requestdetail</literal> for the server
+ functionality.
+ <literal>zoom</literal> for the zoom client api.
+ <literal>ztest</literal> for the simple test server.
+ <literal>malloc, nmem, odr, eventl</literal> for internal
+ debugging of yaz itself.
+ Of course, any program using yaz is welcome to define as many new
+ ones, as it needs.
+ </para>
+ <para>
+ By default the log is written to stderr, but this can be changed by a call
+ to <function>yaz_log_init_file</function> or
+ <function>yaz_log_init</function>. If the log is directed to a file, the
+ file size is checked at every write, and if it exceeds the limit given in
+ <function>yaz_log_init_max_size</function>, the log is rotated. The
+ rotation keeps one old version (with a <literal>.1</literal> appended to
+ the name). The size defaults to 1GB. Setting it to zero will disable the
+ rotation feature.
+ </para>
+ <screen>
+ A typical yaz-log looks like this
+ 13:23:14-23/11 yaz-ztest(1) [session] Starting session from tcp:127.0.0.1 (pid=30968)
+ 13:23:14-23/11 yaz-ztest(1) [request] Init from 'YAZ' (81) (ver 2.0.28) OK
+ 13:23:17-23/11 yaz-ztest(1) [request] Search Z: @attrset Bib-1 foo OK:7 hits
+ 13:23:22-23/11 yaz-ztest(1) [request] Present: [1] 2+2 OK 2 records returned
+ 13:24:13-23/11 yaz-ztest(1) [request] Close OK
+ </screen>
+ <para>
+ The log entries start with a time stamp. This can be omitted by setting the
+ <literal>YLOG_NOTIME</literal> bit in the loglevel. This way automatic tests
+ can be hoped to produce identical log files, that are easy to diff. The
+ format of the time stamp can be set with
+ <function>yaz_log_time_format</function>, which takes a format string just
+ like <function>strftime</function>.
+ </para>
+ <para>
+ Next in a log line comes the prefix, often the name of the program. For
+ yaz-based servers, it can also contain the session number. Then
+ comes one or more logbits in square brackets, depending on the logging
+ level set by <function>yaz_log_init_level</function> and the loglevel
+ passed to <function>yaz_log_init_level</function>. Finally comes the format
+ string and additional values passed to <function>yaz_log</function>
+ </para>
+ <para>
+ The log level <literal>YLOG_LOGLVL</literal>, enabled by the string
+ <literal>loglevel</literal>, will log all the log-level affecting
+ operations. This can come in handy if you need to know what other log
+ levels would be useful. Grep the logfile for <literal>[loglevel]</literal>.
+ </para>
+ <para>
+ The log system is almost independent of the rest of &yaz;, the only
+ important dependence is of <filename>nmem</filename>, and that only for
+ using the semaphore definition there.
+ </para>
+ <para>
+ The dynamic log levels and log rotation were introduced in &yaz; 2.0.28. At
+ the same time, the log bit names were changed from
+ <literal>LOG_something</literal> to <literal>YLOG_something</literal>,
+ to avoid collision with <filename>syslog.h</filename>.
+ </para>
+ </sect1>
+ <sect1 id="marc">
+ <title>MARC</title>
+ <para>
+ YAZ provides a fast utility for working with MARC records.
+ Early versions of the MARC utility only allowed decoding of ISO2709.
+ Today the utility may both encode - and decode to a varity of formats.
+ </para>
+ <synopsis><![CDATA[
+ #include <yaz/marcdisp.h>
+
+ /* create handler */
+ yaz_marc_t yaz_marc_create(void);
+ /* destroy */
+ void yaz_marc_destroy(yaz_marc_t mt);
+
+ /* set XML mode YAZ_MARC_LINE, YAZ_MARC_SIMPLEXML, ... */
+ void yaz_marc_xml(yaz_marc_t mt, int xmlmode);
+ #define YAZ_MARC_LINE 0
+ #define YAZ_MARC_SIMPLEXML 1
+ #define YAZ_MARC_OAIMARC 2
+ #define YAZ_MARC_MARCXML 3
+ #define YAZ_MARC_ISO2709 4
+ #define YAZ_MARC_XCHANGE 5
+ #define YAZ_MARC_CHECK 6
+ #define YAZ_MARC_TURBOMARC 7
+ #define YAZ_MARC_JSON 8
+
+ /* supply iconv handle for character set conversion .. */
+ void yaz_marc_iconv(yaz_marc_t mt, yaz_iconv_t cd);
+
+ /* set debug level, 0=none, 1=more, 2=even more, .. */
+ void yaz_marc_debug(yaz_marc_t mt, int level);
+
+ /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
+ On success, result in *result with size *rsize. */
+ int yaz_marc_decode_buf(yaz_marc_t mt, const char *buf, int bsize,
+ const char **result, size_t *rsize);
+
+ /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
+ On success, result in WRBUF */
+ int yaz_marc_decode_wrbuf(yaz_marc_t mt, const char *buf,
+ int bsize, WRBUF wrbuf);
+]]>
+ </synopsis>
+ <note>
+ <para>
+ The synopsis is just a basic subset of all functionality. Refer
+ to the actual header file <filename>marcdisp.h</filename> for
+ details.
+ </para>
+ </note>
+ <para>
+ A MARC conversion handle must be created by using
+ <function>yaz_marc_create</function> and destroyed
+ by calling <function>yaz_marc_destroy</function>.
+ </para>
+ <para>
+ All other function operate on a <literal>yaz_marc_t</literal> handle.
+ The output is specified by a call to <function>yaz_marc_xml</function>.
+ The <literal>xmlmode</literal> must be one of
+ <variablelist>
+ <varlistentry>
+ <term>YAZ_MARC_LINE</term>
+ <listitem>
+ <para>
+ A simple line-by-line format suitable for display but not
+ recommend for further (machine) processing.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_MARCXML</term>
+ <listitem>
+ <para>
+ <ulink url="&url.marcxml;">MARCXML</ulink>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_ISO2709</term>
+ <listitem>
+ <para>
+ ISO2709 (sometimes just referred to as "MARC").
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_XCHANGE</term>
+ <listitem>
+ <para>
+ <ulink url="&url.marcxchange;">MarcXchange</ulink>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_CHECK</term>
+ <listitem>
+ <para>
+ Pseudo format for validation only. Does not generate
+ any real output except diagnostics.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_TURBOMARC</term>
+ <listitem>
+ <para>
+ XML format with same semantics as MARCXML but more compact
+ and geared towards fast processing with XSLT. Refer to
+ <xref linkend="tools.turbomarc"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_JSON</term>
+ <listitem>
+ <para>
+ <ulink url="&url.marc_in_json;">MARC-in_JSON</ulink> format.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ The actual conversion functions are
+ <function>yaz_marc_decode_buf</function> and
+ <function>yaz_marc_decode_wrbuf</function> which decodes and encodes
+ a MARC record. The former function operates on simple buffers, the
+ stores the resulting record in a WRBUF handle (WRBUF is a simple string
+ type).
+ </para>
+ <example id="example.marc.display">
+ <title>Display of MARC record</title>
+ <para>
+ The following program snippet illustrates how the MARC API may
+ be used to convert a MARC record to the line-by-line format:
+ <programlisting><![CDATA[
+ void print_marc(const char *marc_buf, int marc_buf_size)
+ {
+ char *result; /* for result buf */
+ size_t result_len; /* for size of result */
+ yaz_marc_t mt = yaz_marc_create();
+ yaz_marc_xml(mt, YAZ_MARC_LINE);
+ yaz_marc_decode_buf(mt, marc_buf, marc_buf_size,
+ &result, &result_len);
+ fwrite(result, result_len, 1, stdout);
+ yaz_marc_destroy(mt); /* note that result is now freed... */
+ }
+]]>
+ </programlisting>
+ </para>
+ </example>
+ <sect2 id="tools.turbomarc">
+ <title>TurboMARC</title>
+ <para>
+ TurboMARC is yet another XML encoding of a MARC record. The format
+ was designed for fast processing with XSLT.
+ </para>
+ <para>
+ Applications like
+ Pazpar2 uses XSLT to convert an XML encoded MARC record to an internal
+ representation. This conversion mostly check the tag of a MARC field
+ to determine the basic rules in the conversion. This check is
+ costly when that is tag is encoded as an attribute in MARCXML.
+ By having the tag value as the element instead, makes processing
+ many times faster (at least for Libxslt).
+ </para>
+ <para>
+ TurboMARC is encoded as follows:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Record elements is part of namespace
+ "<literal>http://www.indexdata.com/turbomarc</literal>".
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A record is enclosed in element <literal>r</literal>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A collection of records is enclosed in element
+ <literal>collection</literal>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The leader is encoded as element <literal>l</literal> with the
+ leader content as its (text) value.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A control field is encoded as element <literal>c</literal> concatenated
+ with the tag value of the control field if the tag value
+ matches the regular expression <literal>[a-zA-Z0-9]*</literal>.
+ If the tag value do not match the regular expression
+ <literal>[a-zA-Z0-9]*</literal> the control field is encoded
+ as element <literal>c</literal> and attribute <literal>code</literal>
+ will hold the tag value.
+ This rule ensure that in the rare cases where a tag value might
+ result in a non-wellformed XML YAZ encode it as a coded attribute
+ (as in MARCXML).
+ </para>
+ <para>
+ The control field content is the the text value of this element.
+ Indicators are encoded as attribute names
+ <literal>i1</literal>, <literal>i2</literal>, etc.. and
+ corresponding values for each indicator.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A data field is encoded as element <literal>d</literal> concatenated
+ with the tag value of the data field or using the attribute
+ <literal>code</literal> as described in the rules for control fields.
+ The children of the data field element is subfield elements.
+ Each subfield element is encoded as <literal>s</literal>
+ concatenated with the sub field code.
+ The text of the subfield element is the contents of the subfield.
+ Indicators are encoded as attributes for the data field element similar
+ to the encoding for control fields.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="tools.retrieval">
+ <title>Retrieval Facility</title>
+ <para>
+ YAZ version 2.1.20 or later includes a Retrieval facility tool
+ which allows a SRU/Z39.50 to describe itself and perform record
+ conversions. The idea is the following:
+ <itemizedlist>
+ <listitem>
+ <para>
+ An SRU/Z39.50 client sends a retrieval request which includes
+ a combination of the following parameters: syntax (format),
+ schema (or element set name).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The retrieval facility is invoked with parameters in a
+ server/proxy. The retrieval facility matches the parameters a set of
+ "supported" retrieval types.
+ If there is no match, the retrieval signals an error
+ (syntax and / or schema not supported).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ For a successful match, the backend is invoked with the same
+ or altered retrieval parameters (syntax, schema). If
+ a record is received from the backend, it is converted to the
+ frontend name / syntax.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The resulting record is sent back the client and tagged with
+ the frontend syntax / schema.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The Retrieval facility is driven by an XML configuration. The
+ configuration is neither Z39.50 ZeeRex or SRU ZeeRex. But it
+ should be easy to generate both of them from the XML configuration.
+ (unfortunately the two versions
+ of ZeeRex differ substantially in this regard).
+ </para>
+ <sect2 id="tools.retrieval.format">
+ <title>Retrieval XML format</title>
+ <para>
+ All elements should be covered by namespace
+ <literal>http://indexdata.com/yaz</literal> .
+ The root element node must be <literal>retrievalinfo</literal>.
+ </para>
+ <para>
+ The <literal>retrievalinfo</literal> must include one or
+ more <literal>retrieval</literal> elements. Each
+ <literal>retrieval</literal> defines specific combination of
+ syntax, name and identifier supported by this retrieval service.
+ </para>
+ <para>
+ The <literal>retrieval</literal> element may include any of the
+ following attributes:
+ <variablelist>
+ <varlistentry><term><literal>syntax</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Defines the record syntax. Possible values is any
+ of the names defined in YAZ' OID database or a raw
+ OID in (n.n ... n).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term><literal>name</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Defines the name of the retrieval format. This can be
+ any string. For SRU, the value, is equivalent to schema (short-hand);
+ for Z39.50 it's equivalent to simple element set name.
+ For YAZ 3.0.24 and later this name may be specified as a glob
+ expression with operators
+ <literal>*</literal> and <literal>?</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term><literal>identifier</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Defines the URI schema name of the retrieval format. This can be
+ any string. For SRU, the value, is equivalent to URI schema.
+ For Z39.50, there is no equivalent.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ The <literal>retrieval</literal> may include one
+ <literal>backend</literal> element. If a <literal>backend</literal>
+ element is given, it specifies how the records are retrieved by
+ some backend and how the records are converted from the backend to
+ the "frontend".
+ </para>
+ <para>
+ The attributes, <literal>name</literal> and <literal>syntax</literal>
+ may be specified for the <literal>backend</literal> element. These
+ semantics of these attributes is equivalent to those for the
+ <literal>retrieval</literal>. However, these values are passed to
+ the "backend".
+ </para>
+ <para>
+ The <literal>backend</literal> element may includes one or more
+ conversion instructions (as children elements). The supported
+ conversions are:
+ <variablelist>
+ <varlistentry><term><literal>marc</literal></term>
+ <listitem>
+ <para>
+ The <literal>marc</literal> element specifies a conversion
+ to - and from ISO2709 encoded MARC and
+ <ulink url="&url.marcxml;">&acro.marcxml;</ulink>/MarcXchange.
+ The following attributes may be specified:
+ <variablelist>
+ <varlistentry>
+ <term><literal>inputformat</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Format of input. Supported values are
+ <literal>marc</literal> (for ISO2709), <literal>xml</literal>
+ (MARCXML/MarcXchange) and <literal>json</literal>
+ (<ulink url="&url.marc_in_json;">MARC-in_JSON</ulink>).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>outputformat</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Format of output. Supported values are
+ <literal>line</literal> (MARC line format);
+ <literal>marcxml</literal> (for MARCXML),
+ <literal>marc</literal> (ISO2709),
+ <literal>marcxhcange</literal> (for MarcXchange),
+ or <literal>json</literal>
+ (<ulink url="&url.marc_in_json;">MARC-in_JSON </ulink>).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>inputcharset</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Encoding of input. For XML input formats, this need not
+ be given, but for ISO2709 based inputformats, this should
+ be set to the encoding used. For MARC21 records, a common
+ inputcharset value would be <literal>marc-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>outputcharset</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Encoding of output. If outputformat is XML based, it is
+ strongly recommened to use <literal>utf-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>xslt</literal></term>
+ <listitem>
+ <para>
+ The <literal>xslt</literal> element specifies a conversion
+ via &acro.xslt;. The following attributes may be specified:
+ <variablelist>
+ <varlistentry><term><literal>stylesheet</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Stylesheet file.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>solrmarc</literal></term>
+ <listitem>
+ <para>
+ The <literal>solrmarc</literal> decodes solrmarc records.
+ It assumes that the input is pure solrmarc text (no escaping)
+ and will convert all sequences of the form #XX; to a single
+ character of the hexadecimal value as given by XX. The output,
+ presumably, is a valid ISO2709 buffer.
+ </para>
+ <para>
+ This conversion is available in YAZ 5.0.21 and later.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+ <sect2 id="tools.retrieval.examples">
+ <title>Retrieval Facility Examples</title>
+ <example id="tools.retrieval.marc21">
+ <title>MARC21 backend</title>
+ <para>
+ A typical way to use the retrieval facility is to enable XML
+ for servers that only supports ISO2709 encoded MARC21 records.
+ </para>
+ <programlisting><![CDATA[
+ <retrievalinfo>
+ <retrieval syntax="usmarc" name="F"/>
+ <retrieval syntax="usmarc" name="B"/>
+ <retrieval syntax="xml" name="marcxml"
+ identifier="info:srw/schema/1/marcxml-v1.1">
+ <backend syntax="usmarc" name="F">
+ <marc inputformat="marc" outputformat="marcxml"
+ inputcharset="marc-8"/>
+ </backend>
+ </retrieval>
+ <retrieval syntax="xml" name="dc">
+ <backend syntax="usmarc" name="F">
+ <marc inputformat="marc" outputformat="marcxml"
+ inputcharset="marc-8"/>
+ <xslt stylesheet="MARC21slim2DC.xsl"/>
+ </backend>
+ </retrieval>
+ </retrievalinfo>
+]]>
+ </programlisting>
+ <para>
+ This means that our frontend supports:
+ <itemizedlist>
+ <listitem>
+ <para>
+ MARC21 F(ull) records.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ MARC21 B(rief) records.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ MARCXML records.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Dublin core records.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </example>
+ <example id="tools.retrieval.marcxml">
+ <title>MARCXML backend</title>
+ <para>
+ SRW/SRU and Solr backends returns records in XML.
+ If they return MARCXML or MarcXchange, the retrieval module
+ can convert those into ISO2709 formats, most commonly USMARC
+ (AKA MARC21).
+ In this example, the backend returns MARCXML for schema="marcxml".
+ </para>
+ <programlisting><![CDATA[
+ <retrievalinfo>
+ <retrieval syntax="usmarc">
+ <backend syntax="xml" name="marcxml">
+ <marc inputformat="xml" outputformat="marc"
+ outputcharset="marc-8"/>
+ </backend>
+ </retrieval>
+ <retrieval syntax="xml" name="marcxml"
+ identifier="info:srw/schema/1/marcxml-v1.1"/>
+ <retrieval syntax="xml" name="dc">
+ <backend syntax="xml" name="marcxml">
+ <xslt stylesheet="MARC21slim2DC.xsl"/>
+ </backend>
+ </retrieval>
+ </retrievalinfo>
+]]>
+ </programlisting>
+ <para>
+ This means that our frontend supports:
+ <itemizedlist>
+ <listitem>
+ <para>
+ MARC21 records (any element set name) in MARC-8 encoding.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ MARCXML records for element-set=marcxml
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Dublin core records for element-set=dc.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="tools.retrieval.api">
+ <title>API</title>
+ <para>
+ It should be easy to use the retrieval systems from applications. Refer
+ to the headers
+ <filename>yaz/retrieval.h</filename> and
+ <filename>yaz/record_conv.h</filename>.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="sorting">
+ <title>Sorting</title>
+ <para>
+ This chapter describes sorting and how it is supported in YAZ.
+ Sorting applies to a result-set.
+ The
+ <ulink url="http://www.loc.gov/z3950/agency/markup/05.html#3.2.7">
+ Z39.50 sorting facility
+ </ulink>
+ takes one or more input result-sets
+ and one result-set as output. The most simple case is that
+ the input-set is the same as the output-set.
+ </para>
+ <para>
+ Z39.50 sorting has a separate APDU (service) that is, thus, performed
+ following a search (two phases).
+ </para>
+ <para>
+ In SRU/Solr, however, the model is different. Here, sorting is specified
+ during the the search operation. Note, however, that SRU might
+ perform sort as separate search, by referring to an existing result-set
+ in the query (result-set reference).
+ </para>
+ <sect2>
+ <title>Using the Z39.50 sort service</title>
+ <para>
+ yaz-client and the ZOOM API supports the Z39.50 sort facility. In any
+ case the sort sequence or sort critiera is using a string notation.
+ This notation is a one-line notation suitable for being manually
+ entered or generated and allows for easy logging (one liner).
+ For the ZOOM API, the sort is specified in the call to ZOOM_query_sortby
+ function. For yaz-client the sort is performed and specified using
+ the sort and sort+ commands. For description of the sort criteria notation
+ refer to the <link linkend="sortspec">sort command</link> in the
+ yaz-client manual.
+ </para>
+ <para>
+ The ZOOM API might choose one of several sort strategies for
+ sorting. Refer to <xref linkend="zoom-sort-strategy"/>.
+ </para>
+ </sect2>
+ <sect2>
+ <title>Type-7 sort</title>
+ <para>
+ Type-7 sort is an extension to the Bib-1 based RPN query where the
+ sort specification is embedded as an Attribute-Plus-Term.
+ </para>
+ <para>
+ The objectives for introducing Type-7 sorting is that it allows
+ a client to perform sorting even if it does not implement/support
+ Z39.50 sort. Virtually all Z39.50 client software supports
+ RPN queries. It also may improve performance because the sort
+ critieria is specified along with the search query.
+ </para>
+ <para>
+ The sort is triggered by the presence of type 7 and the value of type 7
+ specifies the
+ <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortKeySpec">
+ sortRelation
+ </ulink>
+ The value for type 7 is 1 for ascending and 2 for descending.
+ For the
+ <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortElement">
+ sortElement
+ </ulink>
+ only the generic part is handled. If generic sortKey is of type
+ sortField, then attribute type 1 is present and the value is
+ sortField (InternationalString). If generic sortKey is of type
+ sortAttributes, then the attributes in list is used . generic sortKey
+ of type elementSpec is not supported.
+ </para>
+ <para>
+ The term in the sorting Attribute-Plus-Term combo should hold
+ an integer. The value is 0 for primary sorting criteria, 1 for second
+ criteria, etc.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="facets">
+ <title>Facets</title>
+ <para>
+ YAZ supports facets for in Solr, SRU 2.0 and Z39.50 protocols.
+ </para>
+ <para>
+ Like Type-1/RPN, YAZ supports a string notation for specifying
+ facets. For the API this is performed by
+ <function>yaz_pqf_parse_facet_list</function>.
+ </para>
+ <para>
+ For ZOOM C the facets are given by option "facets"
+ For yaz-client it is used for the facets command.
+ </para>
+ <para>
+ The grammar of this specification is as follows:
+ <literallayout>
+ facet-spec ::= facet-list
+
+ facet-list ::= facet-list ',' attr-spec | attr-spec
+
+ attr-spec ::= attr-spec '@attr' string | '@attr' string
+
+ </literallayout>
+ The notation is inspired by PQF. The string following '@attr'
+ may not include blanks and is of the form
+ <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>,
+ where <replaceable>type</replaceable> is an integer and
+ <replaceable>value</replaceable> is a string or an integer.
+ </para>
+ <para>
+ The Facets specification is not Bib-1. The following types apply:
+ </para>
+ <table id="facet.attributes">
+ <title>Facet attributes</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="type"></colspec>
+ <colspec colwidth="9*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>1</entry>
+ <entry>
+ Field-name. This is often a string, eg "Author", "Year", etc.
+ </entry>
+ </row>
+ <row>
+ <entry>2</entry>
+ <entry>
+ Sort order. Value should be an integer.
+ Value 0: count descending (frequency). Value 1: alpha ascending.
+ </entry>
+ </row>
+ <row>
+ <entry>3</entry>
+ <entry>
+ Number of terms requested.
+ </entry>
+ </row>
+ <row>
+ <entry>4</entry>
+ <entry>
+ Start offset.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ </chapter>
+ <chapter id="odr">
+ <title>The ODR Module</title>
+ <sect1 id="odr.introduction">
+ <title>Introduction</title>
+ <para>
+ &odr; is the BER-encoding/decoding subsystem of &yaz;. Care as been taken
+ to isolate &odr; from the rest of the package - specifically from the
+ transport interface. &odr; may be used in any context where basic
+ ASN.1/BER representations are used.
+ </para>
+ <para>
+ If you are only interested in writing a Z39.50 implementation based on
+ the PDUs that are already provided with &yaz;, you only need to concern
+ yourself with the section on managing ODR streams
+ (<xref linkend="odr.use"/>). Only if you need to
+ implement ASN.1 beyond that which has been provided, should you
+ worry about the second half of the documentation
+ (<xref linkend="odr.programming"/>).
+ If you use one of the higher-level interfaces, you can skip this
+ section entirely.
+ </para>
+ <para>
+ This is important, so we'll repeat it for emphasis: <emphasis>You do
+ not need to read <xref linkend="odr.programming"/>
+ to implement Z39.50 with &yaz;.</emphasis>
+ </para>
+ <para>
+ If you need a part of the protocol that isn't already in &yaz;, you
+ should contact the authors before going to work on it yourself: We
+ might already be working on it. Conversely, if you implement a useful
+ part of the protocol before us, we'd be happy to include it in a
+ future release.
+ </para>
+ </sect1>
+ <sect1 id="odr.use">
+ <title>Using ODR</title>
+ <sect2 id="odr.streams">
+ <title>ODR Streams</title>
+ <para>
+ Conceptually, the ODR stream is the source of encoded data in the
+ decoding mode; when encoding, it is the receptacle for the encoded
+ data. Before you can use an ODR stream it must be allocated. This is
+ done with the function
+ </para>
+ <synopsis>
+ ODR odr_createmem(int direction);
+ </synopsis>
+ <para>
+ The <function>odr_createmem()</function> function takes as argument one
+ of three manifest constants: <literal>ODR_ENCODE</literal>,
+ <literal>ODR_DECODE</literal>, or <literal>ODR_PRINT</literal>.
+ An &odr; stream can be in only one mode - it is not possible to change
+ its mode once it's selected. Typically, your program will allocate
+ at least two ODR streams - one for decoding, and one for encoding.
+ </para>
+ <para>
+ When you're done with the stream, you can use
+ </para>
+ <synopsis>
+ void odr_destroy(ODR o);
+ </synopsis>
+ <para>
+ to release the resources allocated for the stream.
+ </para>
+ </sect2>
+ <sect2 id="odr.memory.management">
+ <title id="memory">Memory Management</title>
+ <para>
+ Two forms of memory management take place in the &odr; system. The first
+ one, which has to do with allocating little bits of memory (sometimes
+ quite large bits of memory, actually) when a protocol package is
+ decoded, and turned into a complex of interlinked structures. This
+ section deals with this system, and how you can use it for your own
+ purposes. The next section deals with the memory management which is
+ required when encoding data - to make sure that a large enough buffer is
+ available to hold the fully encoded PDU.
+ </para>
+ <para>
+ The &odr; module has its own memory management system, which is
+ used whenever memory is required. Specifically, it is used to allocate
+ space for data when decoding incoming PDUs. You can use the memory
+ system for your own purposes, by using the function
+ </para>
+ <synopsis>
+ void *odr_malloc(ODR o, size_t size);
+ </synopsis>
+ <para>
+ You can't use the normal <function>free(2)</function> routine to free
+ memory allocated by this function, and &odr; doesn't provide a parallel
+ function. Instead, you can call
+ </para>
+ <synopsis>
+ void odr_reset(ODR o);
+ </synopsis>
+ <para>
+ when you are done with the
+ memory: Everything allocated since the last call to
+ <function>odr_reset()</function> is released.
+ The <function>odr_reset()</function> call is also required to clear
+ up an error condition on a stream.
+ </para>
+ <para>
+ The function
+ </para>
+ <synopsis>
+ size_t odr_total(ODR o);
+ </synopsis>
+ <para>
+ returns the number of bytes allocated on the stream since the last call to
+ <function>odr_reset()</function>.
+ </para>
+ <para>
+ The memory subsystem of &odr; is fairly efficient at allocating and
+ releasing little bits of memory. Rather than managing the individual,
+ small bits of space, the system maintains a free-list of larger chunks
+ of memory, which are handed out in small bits. This scheme is
+ generally known as a <emphasis>nibble memory</emphasis> system.
+ It is very useful for maintaining short-lived constructions such
+ as protocol PDUs.
+ </para>
+ <para>
+ If you want to retain a bit of memory beyond the next call to
+ <function>odr_reset()</function>, you can use the function
+ </para>
+ <synopsis>
+ ODR_MEM odr_extract_mem(ODR o);
+ </synopsis>
+ <para>
+ This function will give you control of the memory recently allocated
+ on the ODR stream. The memory will live (past calls to
+ <function>odr_reset()</function>), until you call the function
+ </para>
+ <synopsis>
+ void odr_release_mem(ODR_MEM p);
+ </synopsis>
+ <para>
+ The opaque <literal>ODR_MEM</literal> handle has no other purpose than
+ referencing the memory block for you until you want to release it.
+ </para>
+ <para>
+ You can use <function>odr_extract_mem()</function> repeatedly between
+ allocating data, to retain individual control of separate chunks of data.
+ </para>
+ </sect2>
+ <sect2 id="odr.encoding.and.decoding">
+ <title>Encoding and Decoding Data</title>
+ <para>
+ When encoding data, the ODR stream will write the encoded octet string
+ in an internal buffer. To retrieve the data, use the function
+ </para>
+ <synopsis>
+ char *odr_getbuf(ODR o, int *len, int *size);
+ </synopsis>
+ <para>
+ The integer pointed to by len is set to the length of the encoded
+ data, and a pointer to that data is returned. <literal>*size</literal>
+ is set to the size of the buffer (unless <literal>size</literal> is null,
+ signaling that you are not interested in the size). The next call to
+ a primitive function using the same &odr; stream will overwrite the
+ data, unless a different buffer has been supplied using the call
+ </para>
+ <synopsis>
+ void odr_setbuf(ODR o, char *buf, int len, int can_grow);
+ </synopsis>
+ <para>
+ which sets the encoding (or decoding) buffer used by
+ <literal>o</literal> to <literal>buf</literal>, using the length
+ <literal>len</literal>.
+ Before a call to an encoding function, you can use
+ <function>odr_setbuf()</function> to provide the stream with an encoding
+ buffer of sufficient size (length). The <literal>can_grow</literal>
+ parameter tells the encoding &odr; stream whether it is allowed to use
+ <function>realloc(2)</function> to increase the size of the buffer when
+ necessary. The default condition of a new encoding stream is equivalent
+ to the results of calling
+ </para>
+ <synopsis>
+ odr_setbuf(stream, 0, 0, 1);
+ </synopsis>
+ <para>
+ In this case, the stream will allocate and reallocate memory as
+ necessary. The stream reallocates memory by repeatedly doubling the
+ size of the buffer - the result is that the buffer will typically
+ reach its maximum, working size with only a small number of reallocation
+ operations. The memory is freed by the stream when the latter is destroyed,
+ unless it was assigned by the user with the <literal>can_grow</literal>
+ parameter set to zero (in this case, you are expected to retain
+ control of the memory yourself).
+ </para>
+ <para>
+ To assume full control of an encoded buffer, you must first call
+ <function>odr_getbuf()</function> to fetch the buffer and its length.
+ Next, you should call <function>odr_setbuf()</function> to provide a
+ different buffer (or a null pointer) to the stream. In the simplest
+ case, you will reuse the same buffer over and over again, and you
+ will just need to call <function>odr_getbuf()</function> after each
+ encoding operation to get the length and address of the buffer.
+ Note that the stream may reallocate the buffer during an encoding
+ operation, so it is necessary to retrieve the correct address after
+ each encoding operation.
+ </para>
+ <para>
+ It is important to realize that the ODR stream will not release this
+ memory when you call <function>odr_reset()</function>: It will
+ merely update its internal pointers to prepare for the encoding of a
+ new data value.
+ When the stream is released by the <function>odr_destroy()</function>
+ function, the memory given to it by <function>odr_setbuf</function> will
+ be released <emphasis>only</emphasis> if the <literal>can_grow</literal>
+ parameter to <function>odr_setbuf()</function> was nonzero. The
+ <literal>can_grow</literal> parameter, in other words, is a way of
+ signaling who is to own the buffer, you or the ODR stream. If you never call
+ <function>odr_setbuf()</function> on your encoding stream, which is
+ typically the case, the buffer allocated by the stream will belong to
+ the stream by default.
+ </para>
+ <para>
+ When you wish to decode data, you should first call
+ <function>odr_setbuf()</function>, to tell the decoding stream
+ where to find the encoded data, and how long the buffer is
+ (the <literal>can_grow</literal> parameter is ignored by a decoding
+ stream). After this, you can call the function corresponding to the
+ data you wish to decode (eg, <function>odr_integer()</function> odr
+ <function>z_APDU()</function>).
+ </para>
+ <example id="example.odr.encoding.and.decoding.functions">
+ <title>Encoding and decoding functions</title>
+ <synopsis>
+ int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
+
+ int z_APDU(ODR o, Z_APDU **p, int optional, const char *name);
+ </synopsis>
+ </example>
+ <para>
+ If the data is absent (or doesn't match the tag corresponding to
+ the type), the return value will be either 0 or 1 depending on the
+ <literal>optional</literal> flag. If <literal>optional</literal>
+ is 0 and the data is absent, an error flag will be raised in the
+ stream, and you'll need to call <function>odr_reset()</function> before
+ you can use the stream again. If <literal>optional</literal> is
+ nonzero, the pointer <emphasis>pointed</emphasis> to/ by
+ <literal>p</literal> will be set to the null value, and the function
+ will return 1.
+ The <literal>name</literal> argument is used to pretty-print the
+ tag in question. It may be set to <literal>NULL</literal> if
+ pretty-printing is not desired.
+ </para>
+ <para>
+ If the data value is found where it's expected, the pointer
+ <emphasis>pointed to</emphasis> by the <literal>p</literal> argument
+ will be set to point to the decoded type.
+ The space for the type will be allocated and owned by the &odr;
+ stream, and it will live until you call
+ <function>odr_reset()</function> on the stream. You cannot use
+ <function>free(2)</function> to release the memory.
+ You can decode several data elements (by repeated calls to
+ <function>odr_setbuf()</function> and your decoding function), and
+ new memory will be allocated each time. When you do call
+ <function>odr_reset()</function>, everything decoded since the
+ last call to <function>odr_reset()</function> will be released.
+ </para>
+ <example id="example.odr.encoding.of.integer">
+ <title>Encoding and decoding of an integer</title>
+ <para>
+ The use of the double indirection can be a little confusing at first
+ (its purpose will become clear later on, hopefully),
+ so an example is in order. We'll encode an integer value, and
+ immediately decode it again using a different stream. A useless, but
+ informative operation.
+ </para>
+ <programlisting><![CDATA[
+void do_nothing_useful(Odr_int value)
+{
+ ODR encode, decode;
+ Odr_int *valp, *resvalp;
+ char *bufferp;
+ int len;
+
+ /* allocate streams */
+ if (!(encode = odr_createmem(ODR_ENCODE)))
+ return;
+ if (!(decode = odr_createmem(ODR_DECODE)))
+ return;
+
+ valp = &value;
+ if (odr_integer(encode, &valp, 0, 0) == 0)
+ {
+ printf("encoding went bad\n");
+ return;
+ }
+ bufferp = odr_getbuf(encode, &len, 0);
+ printf("length of encoded data is %d\n", len);
+
+ /* now let's decode the thing again */
+ odr_setbuf(decode, bufferp, len, 0);
+ if (odr_integer(decode, &resvalp, 0, 0) == 0)
+ {
+ printf("decoding went bad\n");
+ return;
+ }
+ /* ODR_INT_PRINTF format for printf (such as %d) */
+ printf("the value is " ODR_INT_PRINTF "\n", *resvalp);
+
+ /* clean up */
+ odr_destroy(encode);
+ odr_destroy(decode);
+}
+]]>
+ </programlisting>
+ <para>
+ This looks like a lot of work, offhand. In practice, the &odr; streams
+ will typically be allocated once, in the beginning of your program
+ (or at the beginning of a new network session), and the encoding
+ and decoding will only take place in a few, isolated places in your
+ program, so the overhead is quite manageable.
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="odr.printing">
+ <title>Printing</title>
+ <para>
+ When an ODR stream is created of type <literal>ODR_PRINT</literal>
+ the ODR module will print the contents of a PDU in a readable format.
+ By default output is written to the <literal>stderr</literal> stream.
+ This behavior can be changed, however, by calling the function
+ <synopsis>
+ odr_setprint(ODR o, FILE *file);
+ </synopsis>
+ before encoders or decoders are being invoked.
+ It is also possible to direct the output to a buffer (of indeed
+ another file), by using the more generic mechanism:
+ <synopsis>
+ void odr_set_stream(ODR o, void *handle,
+ void (*stream_write)(ODR o, void *handle, int type,
+ const char *buf, int len),
+ void (*stream_close)(void *handle));
+ </synopsis>
+ Here the user provides an opaque handle and two handlers,
+ <replaceable>stream_write</replaceable> for writing,
+ and <replaceable>stream_close</replaceable> which is supposed
+ to close/free resources associated with handle.
+ The <replaceable>stream_close</replaceable> handler is optional and
+ if NULL for the function is provided, it will not be invoked.
+ The <replaceable>stream_write</replaceable> takes the ODR handle
+ as parameter, the user defined handle, a type
+ <literal>ODR_OCTETSTRING</literal>, <literal>ODR_VISIBLESTRING</literal>
+ which indicates the type of contents is being written.
+ </para>
+ <para>
+ Another utility useful for diagnostics (error handling) or as
+ part of the printing facilities is:
+ <synopsis>
+ const char **odr_get_element_path(ODR o);
+ </synopsis>
+ which returns a list of current elements that ODR deals with at the
+ moment. For the returned array, say <literal>ar</literal>,
+ <literal>ar[0]</literal> is the top level element,
+ <literal>ar[n]</literal> is the last. The last element has the
+ property that <literal>ar[n+1] == NULL</literal>.
+ </para>
+ <example id="example.odr.element.path.record">
+ <title>Element Path for record</title>
+ <para>
+ For a database record part of a PresentResponse the
+ array returned by <function>odr_get_element</function>
+ is <literal>presentResponse</literal>, <literal>databaseOrSurDiagnostics</literal>, <literal>?</literal>, <literal>record</literal>, <literal>?</literal>, <literal>databaseRecord</literal> . The question mark appears due to
+ unnamed constructions.
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="odr.diagnostics">
+ <title>Diagnostics</title>
+ <para>
+ The encoding/decoding functions all return 0 when an error occurs.
+ Until you call <function>odr_reset()</function>, you cannot use the
+ stream again, and any function called will immediately return 0.
+ </para>
+ <para>
+ To provide information to the programmer or administrator, the function
+ </para>
+ <synopsis>
+ void odr_perror(ODR o, char *message);
+ </synopsis>
+ <para>
+ is provided, which prints the <literal>message</literal> argument to
+ <literal>stderr</literal> along with an error message from the stream.
+ </para>
+ <para>
+ You can also use the function
+ </para>
+ <synopsis>
+ int odr_geterror(ODR o);
+ </synopsis>
+ <para>
+ to get the current error number from the screen. The number will be
+ one of these constants:
+ </para>
+ <table frame="top" id="odr.error.codes">
+ <title>ODR Error codes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>code</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>OMEMORY</entry><entry>Memory allocation failed.</entry>
+ </row>
+ <row>
+ <entry>OSYSERR</entry><entry>A system- or library call has failed.
+ The standard diagnostic variable <literal>errno</literal> should be
+ examined to determine the actual error.</entry>
+ </row>
+ <row>
+ <entry>OSPACE</entry><entry>No more space for encoding.
+ This will only occur when the user has explicitly provided a
+ buffer for an encoding stream without allowing the system to
+ allocate more space.</entry>
+ </row>
+ <row>
+ <entry>OREQUIRED</entry><entry>This is a common protocol error; A
+ required data element was missing during encoding or decoding.</entry>
+ </row>
+ <row>
+ <entry>OUNEXPECTED</entry><entry>An unexpected data element was
+ found during decoding.</entry>
+ </row>
+ <row>
+ <entry>OOTHER</entry><entry>Other error. This is typically an
+ indication of misuse of the &odr; system by the programmer, and also
+ that the diagnostic system isn't as good as it should be, yet.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ The character string array
+ </para>
+ <synopsis>
+ char *odr_errlist[]
+ </synopsis>
+ <para>
+ can be indexed by the error code to obtain a human-readable
+ representation of the problem.
+ </para>
+ </sect2>
+ <sect2 id="odr.summary.and.synopsis">
+ <title>Summary and Synopsis</title>
+ <synopsis>
+ #include <yaz/odr.h>
+
+ ODR odr_createmem(int direction);
+
+ void odr_destroy(ODR o);
+
+ void odr_reset(ODR o);
+
+ char *odr_getbuf(ODR o, int *len, int *size);
+
+ void odr_setbuf(ODR o, char *buf, int len, int can_grow);
+
+ void *odr_malloc(ODR o, int size);
+
+ NMEM odr_extract_mem(ODR o);
+
+ int odr_geterror(ODR o);
+
+ void odr_perror(ODR o, const char *message);
+
+ extern char *odr_errlist[];
+ </synopsis>
+ </sect2>
+ </sect1>
+ <sect1 id="odr.programming">
+ <title>Programming with ODR</title>
+ <para>
+ The API of &odr; is designed to reflect the structure of ASN.1, rather
+ than BER itself. Future releases may be able to represent data in
+ other external forms.
+ </para>
+ <tip>
+ <para>
+ There is an ASN.1 tutorial available at
+ <ulink url="&url.asn.1.tutorial;">this site</ulink>.
+ This site also has standards for ASN.1 (X.680) and BER (X.690)
+ <ulink url="&url.asn.1.standards;">online</ulink>.
+ </para>
+ </tip>
+ <para>
+ The ODR interface is based loosely on that of the Sun Microsystems
+ XDR routines.
+ Specifically, each function which corresponds to an ASN.1 primitive
+ type has a dual function. Depending on the settings of the ODR
+ stream which is supplied as a parameter, the function may be used
+ either to encode or decode data. The functions that can be built
+ using these primitive functions, to represent more complex data types,
+ share this quality. The result is that you only have to enter the
+ definition for a type once - and you have the functionality of encoding,
+ decoding (and pretty-printing) all in one unit.
+ The resulting C source code is quite compact, and is a pretty
+ straightforward representation of the source ASN.1 specification.
+ </para>
+ <para>
+ In many cases, the model of the XDR functions works quite well in this
+ role.
+ In others, it is less elegant. Most of the hassle comes from the optional
+ SEQUENCE members which don't exist in XDR.
+ </para>
+ <sect2 id="odr.primitive.asn1.types">
+ <title>The Primitive ASN.1 Types</title>
+ <para>
+ ASN.1 defines a number of primitive types (many of which correspond
+ roughly to primitive types in structured programming languages, such as C).
+ </para>
+ <sect3 id="odr.integer">
+ <title>INTEGER</title>
+ <para>
+ The &odr; function for encoding or decoding (or printing) the ASN.1
+ INTEGER type looks like this:
+ </para>
+ <synopsis>
+ int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ The <literal>Odr_int</literal> is just a simple integer.
+ </para>
+ <para>
+ This form is typical of the primitive &odr; functions. They are named
+ after the type of data that they encode or decode. They take an &odr;
+ stream, an indirect reference to the type in question, and an
+ <literal>optional</literal> flag (corresponding to the OPTIONAL keyword
+ of ASN.1) as parameters. They all return an integer value of either one
+ or zero.
+ When you use the primitive functions to construct encoders for complex
+ types of your own, you should follow this model as well. This
+ ensures that your new types can be reused as elements in yet more
+ complex types.
+ </para>
+ <para>
+ The <literal>o</literal> parameter should obviously refer to a properly
+ initialized &odr; stream of the right type (encoding/decoding/printing)
+ for the operation that you wish to perform.
+ </para>
+ <para>
+ When encoding or printing, the function first looks at
+ <literal>* p</literal>. If <literal>* p</literal> (the pointer pointed
+ to by <literal>p</literal>) is a null pointer, this is taken to mean that
+ the data element is absent. If the <literal>optional</literal> parameter
+ is nonzero, the function will return one (signifying success) without
+ any further processing. If the <literal>optional</literal> is zero, an
+ internal error flag is set in the &odr; stream, and the function will
+ return 0. No further operations can be carried out on the stream without
+ a call to the function <function>odr_reset()</function>.
+ </para>
+ <para>
+ If <literal>*p</literal> is not a null pointer, it is expected to
+ point to an instance of the data type. The data will be subjected to
+ the encoding rules, and the result will be placed in the buffer held
+ by the &odr; stream.
+ </para>
+ <para>
+ The other ASN.1 primitives have similar functions that operate in
+ similar manners:
+ </para>
+ </sect3>
+ <sect3 id="odr.boolean">
+ <title>BOOLEAN</title>
+ <synopsis>
+int odr_bool(ODR o, Odr_bool **p, int optional, const char *name);
+ </synopsis>
+ </sect3>
+ <sect3 id="odr.real">
+ <title>REAL</title>
+ <para>
+ Not defined.
+ </para>
+ </sect3>
+ <sect3 id="odr.null">
+ <title>NULL</title>
+ <synopsis>
+int odr_null(ODR o, Odr_null **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ In this case, the value of **p is not important. If <literal>*p</literal>
+ is different from the null pointer, the null value is present, otherwise
+ it's absent.
+ </para>
+ </sect3>
+ <sect3 id="odr.octet.string">
+ <title>OCTET STRING</title>
+ <synopsis>
+typedef struct odr_oct
+{
+ unsigned char *buf;
+ int len;
+} Odr_oct;
+
+int odr_octetstring(ODR o, Odr_oct **p, int optional,
+ const char *name);
+ </synopsis>
+ <para>
+ The <literal>buf</literal> field should point to the character array
+ that holds the octetstring. The <literal>len</literal> field holds the
+ actual length.
+ The character array need not be null terminated.
+ </para>
+ <para>
+ To make things a little easier, an alternative is given for string
+ types that are not expected to contain embedded NULL characters (eg.
+ VisibleString):
+ </para>
+ <synopsis>
+ int odr_cstring(ODR o, char **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ Which encoded or decodes between OCTETSTRING representations and
+ null-terminates C strings.
+ </para>
+ <para>
+ Functions are provided for the derived string types, eg:
+ </para>
+ <synopsis>
+int odr_visiblestring(ODR o, char **p, int optional,
+ const char *name);
+ </synopsis>
+ </sect3>
+ <sect3 id="odr.bit.string">
+ <title>BIT STRING</title>
+ <synopsis>
+int odr_bitstring(ODR o, Odr_bitmask **p, int optional,
+ const char *name);
+ </synopsis>
+ <para>
+ The opaque type <literal>Odr_bitmask</literal> is only suitable for
+ holding relatively brief bit strings, eg. for options fields, etc.
+ The constant <literal>ODR_BITMASK_SIZE</literal> multiplied by 8
+ gives the maximum possible number of bits.
+ </para>
+ <para>
+ A set of macros are provided for manipulating the
+ <literal>Odr_bitmask</literal> type:
+ </para>
+ <synopsis>
+void ODR_MASK_ZERO(Odr_bitmask *b);
+
+void ODR_MASK_SET(Odr_bitmask *b, int bitno);
+
+void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno);
+
+int ODR_MASK_GET(Odr_bitmask *b, int bitno);
+ </synopsis>
+ <para>
+ The functions are modeled after the manipulation functions that
+ accompany the <literal>fd_set</literal> type used by the
+ <function>select(2)</function> call.
+ <literal>ODR_MASK_ZERO</literal> should always be called first on a
+ new bitmask, to initialize the bits to zero.
+ </para>
+ </sect3>
+ <sect3 id="odr.object.identifier">
+ <title>OBJECT IDENTIFIER</title>
+ <synopsis>
+int odr_oid(ODR o, Odr_oid **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ The C OID representation is simply an array of integers, terminated by
+ the value -1 (the <literal>Odr_oid</literal> type is synonymous with
+ the <literal>short</literal> type).
+ We suggest that you use the OID database module (see
+ <xref linkend="tools.oid.database"/>) to handle object identifiers
+ in your application.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2 id="odr.tagging.primitive.types">
+ <title>Tagging Primitive Types</title>
+ <para>
+ The simplest way of tagging a type is to use the
+ <function>odr_implicit_tag()</function> or
+ <function>odr_explicit_tag()</function> macros:
+ </para>
+ <synopsis>
+int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag,
+ int optional, const char *name);
+
+int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag,
+ int optional, const char *name);
+ </synopsis>
+ <para>
+ To create a type derived from the integer type by implicit tagging, you
+ might write:
+ </para>
+ <screen>
+ MyInt ::= [210] IMPLICIT INTEGER
+ </screen>
+ <para>
+ In the &odr; system, this would be written like:
+ </para>
+ <screen>
+int myInt(ODR o, Odr_int **p, int optional, const char *name)
+{
+ return odr_implicit_tag(o, odr_integer, p,
+ ODR_CONTEXT, 210, optional, name);
+}
+ </screen>
+ <para>
+ The function <function>myInt()</function> can then be used like any of
+ the primitive functions provided by &odr;. Note that the behavior of
+ <function>odr_explicit_tag()</function>
+ and <function>odr_implicit_tag()</function> macros
+ act exactly the same as the functions they are applied to - they
+ respond to error conditions, etc, in the same manner - they
+ simply have three extra parameters. The class parameter may
+ take one of the values: <literal>ODR_CONTEXT</literal>,
+ <literal>ODR_PRIVATE</literal>, <literal>ODR_UNIVERSAL</literal>, or
+ <literal>/ODR_APPLICATION</literal>.
+ </para>
+ </sect2>
+ <sect2 id="odr.constructed.types">
+ <title>Constructed Types</title>
+ <para>
+ Constructed types are created by combining primitive types. The
+ &odr; system only implements the SEQUENCE and SEQUENCE OF constructions
+ (although adding the rest of the container types should be simple
+ enough, if the need arises).
+ </para>
+ <para>
+ For implementing SEQUENCEs, the functions
+ </para>
+ <synopsis>
+int odr_sequence_begin(ODR o, void *p, int size, const char *name);
+int odr_sequence_end(ODR o);
+ </synopsis>
+ <para>
+ are provided.
+ </para>
+ <para>
+ The <function>odr_sequence_begin()</function> function should be
+ called in the beginning of a function that implements a SEQUENCE type.
+ Its parameters are the &odr; stream, a pointer (to a pointer to the type
+ you're implementing), and the <literal>size</literal> of the type
+ (typically a C structure). On encoding, it returns 1 if
+ <literal>* p</literal> is a null pointer. The <literal>size</literal>
+ parameter is ignored. On decoding, it returns 1 if the type is found in
+ the data stream. <literal>size</literal> bytes of memory are allocated,
+ and <literal>*p</literal> is set to point to this space.
+ <function>odr_sequence_end()</function> is called at the end of the
+ complex function. Assume that a type is defined like this:
+ </para>
+ <screen>
+MySequence ::= SEQUENCE {
+ intval INTEGER,
+ boolval BOOLEAN OPTIONAL
+}
+ </screen>
+ <para>
+ The corresponding &odr; encoder/decoder function and the associated data
+ structures could be written like this:
+ </para>
+ <screen>
+typedef struct MySequence
+{
+ Odr_int *intval;
+ Odr_bool *boolval;
+} MySequence;
+
+int mySequence(ODR o, MySequence **p, int optional, const char *name)
+{
+ if (odr_sequence_begin(o, p, sizeof(**p), name) == 0)
+ return optional && odr_ok(o);
+ return
+ odr_integer(o, &(*p)->intval, 0, "intval") &&
+ odr_bool(o, &(*p)->boolval, 1, "boolval") &&
+ odr_sequence_end(o);
+}
+ </screen>
+ <para>
+ Note the 1 in the call to <function>odr_bool()</function>, to mark
+ that the sequence member is optional.
+ If either of the member types had been tagged, the macros
+ <function>odr_implicit_tag()</function> or
+ <function>odr_explicit_tag()</function>
+ could have been used.
+ The new function can be used exactly like the standard functions provided
+ with &odr;. It will encode, decode or pretty-print a data value of the
+ <literal>MySequence</literal> type. We like to name types with an
+ initial capital, as done in ASN.1 definitions, and to name the
+ corresponding function with the first character of the name in lower case.
+ You could, of course, name your structures, types, and functions any way
+ you please - as long as you're consistent, and your code is easily readable.
+ <literal>odr_ok</literal> is just that - a predicate that returns the
+ state of the stream. It is used to ensure that the behavior of the new
+ type is compatible with the interface of the primitive types.
+ </para>
+ </sect2>
+ <sect2 id="odr.tagging.constructed.types">
+ <title>Tagging Constructed Types</title>
+ <note>
+ <para>
+ See <xref linkend="odr.tagging.primitive.types"/> for information
+ on how to tag the primitive types, as well as types that are
+ already defined.
+ </para>
+ </note>
+ <sect3 id="odr.implicit.tagging">
+ <title>Implicit Tagging</title>
+ <para>
+ Assume the type above had been defined as
+ </para>
+ <screen>
+MySequence ::= [10] IMPLICIT SEQUENCE {
+ intval INTEGER,
+ boolval BOOLEAN OPTIONAL
+}
+ </screen>
+ <para>
+ You would implement this in &odr; by calling the function
+ </para>
+ <synopsis>
+int odr_implicit_settag(ODR o, int class, int tag);
+ </synopsis>
+ <para>
+ which overrides the tag of the type immediately following it. The
+ macro <function>odr_implicit_tag()</function> works by calling
+ <function>odr_implicit_settag()</function> immediately
+ before calling the function pointer argument.
+ Your type function could look like this:
+ </para>
+ <screen>
+int mySequence(ODR o, MySequence **p, int optional, const char *name)
+{
+ if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 ||
+ odr_sequence_begin(o, p, sizeof(**p), name) == 0)
+ return optional && odr_ok(o);
+ return
+ odr_integer(o, &(*p)->intval, 0, "intval") &&
+ odr_bool(o, &(*p)->boolval, 1, "boolval") &&
+ odr_sequence_end(o);
+}
+ </screen>
+ <para>
+ The definition of the structure <literal>MySequence</literal> would be
+ the same.
+ </para>
+ </sect3>
+ <sect3 id="odr.explicit.tagging">
+ <title>Explicit Tagging</title>
+ <para>
+ Explicit tagging of constructed types is a little more complicated,
+ since you are in effect adding a level of construction to the data.
+ </para>
+ <para>
+ Assume the definition:
+ </para>
+ <screen>
+MySequence ::= [10] IMPLICIT SEQUENCE {
+ intval INTEGER,
+ boolval BOOLEAN OPTIONAL
+}
+ </screen>
+ <para>
+ Since the new type has an extra level of construction, two new functions
+ are needed to encapsulate the base type:
+ </para>
+ <synopsis>
+int odr_constructed_begin(ODR o, void *p, int class, int tag,
+ const char *name);
+
+int odr_constructed_end(ODR o);
+ </synopsis>
+ <para>
+ Assume that the IMPLICIT in the type definition above were replaced
+ with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which
+ would be equivalent). The structure definition would look the same,
+ but the function would look like this:
+ </para>
+ <screen>
+int mySequence(ODR o, MySequence **p, int optional, const char *name)
+{
+ if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0)
+ return optional && odr_ok(o);
+ if (o->direction == ODR_DECODE)
+ *p = odr_malloc(o, sizeof(**p));
+ if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0)
+ {
+ *p = 0; /* this is almost certainly a protocol error */
+ return 0;
+ }
+ return
+ odr_integer(o, &(*p)->intval, 0, "intval") &&
+ odr_bool(o, &(*p)->boolval, 1, "boolval") &&
+ odr_sequence_end(o) &&
+ odr_constructed_end(o);
+}
+ </screen>
+ <para>
+ Notice that the interface here gets kind of nasty. The reason is
+ simple: Explicitly tagged, constructed types are fairly rare in
+ the protocols that we care about, so the
+ esthetic annoyance (not to mention the dangers of a cluttered
+ interface) is less than the time that would be required to develop a
+ better interface. Nevertheless, it is far from satisfying, and it's a
+ point that will be worked on in the future. One option for you would
+ be to simply apply the <function>odr_explicit_tag()</function> macro to
+ the first function, and not
+ have to worry about <function>odr_constructed_*</function> yourself.
+ Incidentally, as you might have guessed, the
+ <function>odr_sequence_</function> functions are themselves
+ implemented using the <function>/odr_constructed_</function> functions.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2 id="odr.sequence.of">
+ <title>SEQUENCE OF</title>
+ <para>
+ To handle sequences (arrays) of a specific type, the function
+ </para>
+ <synopsis>
+int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional),
+ void *p, int *num, const char *name);
+ </synopsis>
+ <para>
+ The <literal>fun</literal> parameter is a pointer to the decoder/encoder
+ function of the type. <literal>p</literal> is a pointer to an array of
+ pointers to your type. <literal>num</literal> is the number of elements
+ in the array.
+ </para>
+ <para>
+ Assume a type
+ </para>
+ <screen>
+MyArray ::= SEQUENCE OF INTEGER
+ </screen>
+ <para>
+ The C representation might be
+ </para>
+ <screen>
+typedef struct MyArray
+{
+ int num_elements;
+ Odr_int **elements;
+} MyArray;
+ </screen>
+ <para>
+ And the function might look like
+ </para>
+ <screen>
+int myArray(ODR o, MyArray **p, int optional, const char *name)
+{
+ if (o->direction == ODR_DECODE)
+ *p = odr_malloc(o, sizeof(**p));
+ if (odr_sequence_of(o, odr_integer, &(*p)->elements,
+ &(*p)->num_elements, name))
+ return 1;
+ *p = 0;
+ return optional && odr_ok(o);
+}
+ </screen>
+ </sect2>
+ <sect2 id="odr.choice.types">
+ <title>CHOICE Types</title>
+ <para>
+ The choice type is used fairly often in some ASN.1 definitions, so
+ some work has gone into streamlining its interface.
+ </para>
+ <para>
+ CHOICE types are handled by the function:
+ </para>
+ <synopsis>
+int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp,
+ const char *name);
+ </synopsis>
+ <para>
+ The <literal>arm</literal> array is used to describe each of the possible
+ types that the CHOICE type may assume. Internally in your application,
+ the CHOICE type is represented as a discriminated union. That is, a
+ C union accompanied by an integer (or enum) identifying the active
+ 'arm' of the union.
+ <literal>whichp</literal> is a pointer to the union discriminator.
+ When encoding, it is examined to determine the current type.
+ When decoding, it is set to reference the type that was found in
+ the input stream.
+ </para>
+ <para>
+ The Odr_arm type is defined thus:
+ </para>
+ <screen>
+typedef struct odr_arm
+{
+ int tagmode;
+ int class;
+ int tag;
+ int which;
+ Odr_fun fun;
+ char *name;
+} Odr_arm;
+ </screen>
+ <para>
+ The interpretation of the fields are:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term>tagmode</term>
+ <listitem><para>Either <literal>ODR_IMPLICIT</literal>,
+ <literal>ODR_EXPLICIT</literal>, or <literal>ODR_NONE</literal> (-1)
+ to mark no tagging.</para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>which</term>
+ <listitem><para>The value of the discriminator that corresponds to
+ this CHOICE element. Typically, it will be a #defined constant, or
+ an enum member.</para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>fun</term>
+ <listitem><para>A pointer to a function that implements the type of
+ the CHOICE member. It may be either a standard &odr; type or a type
+ defined by yourself.</para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>name</term>
+ <listitem><para>Name of tag.</para></listitem>
+ </varlistentry>
+ </variablelist>
+ <para>
+ A handy way to prepare the array for use by the
+ <function>odr_choice()</function> function is to
+ define it as a static, initialized array in the beginning of your
+ decoding/encoding function. Assume the type definition:
+ </para>
+ <screen>
+MyChoice ::= CHOICE {
+ untagged INTEGER,
+ tagged [99] IMPLICIT INTEGER,
+ other BOOLEAN
+}
+ </screen>
+ <para>
+ Your C type might look like
+ </para>
+ <screen>
+typedef struct MyChoice
+{
+ enum
+ {
+ MyChoice_untagged,
+ MyChoice_tagged,
+ MyChoice_other
+ } which;
+ union
+ {
+ Odr_int *untagged;
+ Odr_int *tagged;
+ Odr_bool *other;
+ } u;
+};
+ </screen>
+ <para>
+ And your function could look like this:
+ </para>
+ <screen>
+int myChoice(ODR o, MyChoice **p, int optional, const char *name)
+{
+ static Odr_arm arm[] =
+ {
+ {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"},
+ {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer,
+ "tagged"},
+ {-1, -1, -1, MyChoice_other, odr_boolean, "other"},
+ {-1, -1, -1, -1, 0}
+ };
+
+ if (o->direction == ODR_DECODE)
+ *p = odr_malloc(o, sizeof(**p);
+ else if (!*p)
+ return optional && odr_ok(o);
+
+ if (odr_choice(o, arm, &(*p)->u, &(*p)->which), name)
+ return 1;
+ *p = 0;
+ return optional && odr_ok(o);
+}
+ </screen>
+ <para>
+ In some cases (say, a non-optional choice which is a member of a
+ sequence), you can "embed" the union and its discriminator in the
+ structure belonging to the enclosing type, and you won't need to
+ fiddle with memory allocation to create a separate structure to
+ wrap the discriminator and union.
+ </para>
+ <para>
+ The corresponding function is somewhat nicer in the Sun XDR interface.
+ Most of the complexity of this interface comes from the possibility of
+ declaring sequence elements (including CHOICEs) optional.
+ </para>
+ <para>
+ The ASN.1 specifications naturally requires that each member of a
+ CHOICE have a distinct tag, so they can be told apart on decoding.
+ Sometimes it can be useful to define a CHOICE that has multiple types
+ that share the same tag. You'll need some other mechanism, perhaps
+ keyed to the context of the CHOICE type. In effect, we would like to
+ introduce a level of context-sensitiveness to our ASN.1 specification.
+ When encoding an internal representation, we have no problem, as long
+ as each CHOICE member has a distinct discriminator value. For
+ decoding, we need a way to tell the choice function to look for a
+ specific arm of the table. The function
+ </para>
+ <synopsis>
+void odr_choice_bias(ODR o, int what);
+ </synopsis>
+ <para>
+ provides this functionality. When called, it leaves a notice for the next
+ call to <function>odr_choice()</function> to be called on the decoding
+ stream <literal>o</literal> that only the <literal>arm</literal> entry with
+ a <literal>which</literal> field equal to <literal>what</literal>
+ should be tried.
+ </para>
+ <para>
+ The most important application (perhaps the only one, really) is in
+ the definition of application-specific EXTERNAL encoders/decoders
+ which will automatically decode an ANY member given the direct or
+ indirect reference.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="odr.debugging">
+ <title>Debugging</title>
+ <para>
+ The protocol modules are suffering somewhat from a lack of diagnostic
+ tools at the moment. Specifically ways to pretty-print PDUs that
+ aren't recognized by the system. We'll include something to this end
+ in a not-too-distant release. In the meantime, what we do when we get
+ packages we don't understand is to compile the ODR module with
+ <literal>ODR_DEBUG</literal> defined. This causes the module to dump tracing
+ information as it processes data units. With this output and the
+ protocol specification (Z39.50), it is generally fairly easy to see
+ what goes wrong.
+ </para>
+ </sect1>
+ </chapter>
+ <chapter id="comstack">
+ <title>The COMSTACK Module</title>
+ <sect1 id="comstack.synopsis">
+ <title>Synopsis (blocking mode)</title>
+ <programlisting><![CDATA[
+ COMSTACK stack;
+ char *buf = 0;
+ int size = 0, length_incoming;
+ char server_address_str[] = "localhost:9999";
+ void *server_address_ip;
+ int status;
+
+ char *protocol_package = "GET / HTTP/1.0\r\n\r\n";
+ int protocol_package_length = strlen(protocol_package);
+
+ stack = cs_create(tcpip_type, 1, PROTO_HTTP);
+ if (!stack) {
+ perror("cs_create"); /* use perror() here since we have no stack yet */
+ return -1;
+ }
+
+ server_address_ip = cs_straddr(stack, server_address_str);
+ if (!server_address_ip) {
+ fprintf(stderr, "cs_straddr: address could not be resolved\n");
+ return -1;
+ }
+
+ status = cs_connect(stack, server_address_ip);
+ if (status) {
+ fprintf(stderr, "cs_connect: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ status = cs_rcvconnect(stack);
+ if (status) {
+ fprintf(stderr, "cs_rcvconnect: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ status = cs_put(stack, protocol_package, protocol_package_length);
+ if (status) {
+ fprintf(stderr, "cs_put: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ /* Now get a response */
+ length_incoming = cs_get(stack, &buf, &size);
+ if (!length_incoming) {
+ fprintf(stderr, "Connection closed\n");
+ return -1;
+ } else if (length_incoming < 0) {
+ fprintf(stderr, "cs_get: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ /* Print result */
+ fwrite(buf, length_incoming, 1, stdout);
+
+ /* clean up */
+ cs_close(stack);
+ if (buf)
+ xfree(buf);
+ return 0;
+]]>
+ </programlisting>
+
+ </sect1>
+ <sect1 id="comstack.introduction">
+ <title>Introduction</title>
+ <para>
+ The &comstack;
+ subsystem provides a transparent interface to different types of transport
+ stacks for the exchange of BER-encoded data and HTTP packets.
+ At present, the RFC1729 method (BER over TCP/IP), local UNIX socket and an
+ experimental SSL stack are supported, but others may be added in time.
+ The philosophy of the
+ module is to provide a simple interface by hiding unused options and
+ facilities of the underlying libraries. This is always done at the risk
+ of losing generality, and it may prove that the interface will need
+ extension later on.
+ </para>
+ <note>
+ <para>
+ There hasn't been interest in the XTImOSI stack for some years.
+ Therefore, it is no longer supported.
+ </para>
+ </note>
+ <para>
+ The interface is implemented in such a fashion that only the
+ sub-layers constructed to the transport methods that you wish to
+ use in your application are linked in.
+ </para>
+ <para>
+ You will note that even though simplicity was a goal in the design,
+ the interface is still orders of magnitudes more complex than the
+ transport systems found in many other packages. One reason is that
+ the interface needs to support the somewhat different requirements of
+ the different lower-layer communications stacks; another important
+ reason is that the interface seeks to provide a more or less
+ industrial-strength approach to asynchronous event-handling.
+ When no function is allowed to block, things get more complex -
+ particularly on the server side.
+ We urge you to have a look at the demonstration client and server
+ provided with the package. They are meant to be easily readable and
+ instructive, while still being at least moderately useful.
+ </para>
+ </sect1>
+ <sect1 id="comstack.common">
+ <title>Common Functions</title>
+ <sect2 id="comstack.managing.endpoints">
+ <title>Managing Endpoints</title>
+ <synopsis>
+ COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
+ </synopsis>
+ <para>
+ Creates an instance of the protocol stack - a communications endpoint.
+ The <literal>type</literal> parameter determines the mode
+ of communication. At present the following values are supported:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term><literal>tcpip_type</literal></term>
+ <listitem><para>TCP/IP (BER over TCP/IP or HTTP over TCP/IP)
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>ssl_type</literal></term>
+ <listitem><para>Secure Socket Layer (SSL). This COMSTACK
+ is experimental and is not fully implemented. If
+ HTTP is used, this effectively is HTTPS.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>unix_type</literal></term>
+ <listitem><para>Unix socket (unix only). Local Transfer via
+ file socket. See <citerefentry><refentrytitle>unix</refentrytitle>
+ <manvolnum>7</manvolnum></citerefentry>.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ <para>
+ The <function>cs_create</function> function returns a null-pointer
+ if a system error occurs.
+ The <literal>blocking</literal> parameter should be one if
+ you wish the association to operate in blocking mode, zero otherwise.
+ The <literal>protocol</literal> field should be
+ <literal>PROTO_Z3950</literal> or <literal>PROTO_HTTP</literal>.
+ Protocol <literal>PROTO_SR</literal> is no longer supported.
+ </para>
+ <synopsis>
+ void cs_close(COMSTACK handle);
+ </synopsis>
+ <para>
+ Closes the connection (as elegantly as the lower layers will permit),
+ and releases the resources pointed to by the
+ <literal>handle</literal>
+ parameter. The
+ <literal>handle</literal>
+ should not be referenced again after this call.
+ </para>
+ <note>
+ <para>
+ We really need a soft disconnect, don't we?
+ </para>
+ </note>
+ </sect2>
+ <sect2 id="comstack.data.exchange">
+ <title>Data Exchange</title>
+ <synopsis>
+ int cs_put(COMSTACK handle, char *buf, int len);
+ </synopsis>
+ <para>
+ Sends <literal>buf</literal> down the wire.
+ In blocking mode, this function will return only when a full buffer has
+ been written, or an error has occurred. In nonblocking mode, it's
+ possible that the function will be unable to send the full buffer
+ at once, which will be indicated by a return value of 1.
+ The function will keep track of the number of octets already written; you
+ should call it repeatedly with the same values of <literal>buf</literal>
+ and <literal>len</literal>, until the buffer has been transmitted.
+ When a full buffer has been sent, the function will return 0 for
+ success. -1 indicates an error condition (see below).
+ </para>
+ <synopsis>
+ int cs_get(COMSTACK handle, char **buf, int *size);
+ </synopsis>
+ <para>
+ Receives a PDU or HTTP Response from the peer. Returns the number of
+ bytes read.
+ In nonblocking mode, it is possible that not all of the packet can be
+ read at once. In this case, the function returns 1. To simplify the
+ interface, the function is
+ responsible for managing the size of the buffer. It will be reallocated
+ if necessary to contain large packages, and will sometimes be moved
+ around internally by the subsystem when partial packages are read. Before
+ calling
+ <function>cs_get</function>
+ for the fist time, the buffer can be initialized to the null pointer,
+ and the length should also be set to 0 - cs_get will perform a
+ <function>malloc(2)</function>
+ on the buffer for you. When a full buffer has been read, the size of
+ the package is returned (which will always be greater than 1). -1
+ indicates an error condition.
+ </para>
+ <para>
+ See also the <function>cs_more()</function> function below.
+ </para>
+ <synopsis>
+ int cs_more(COMSTACK handle);
+ </synopsis>
+ <para>
+ The <function>cs_more()</function> function should be used in conjunction
+ with <function>cs_get</function> and
+ <function>select(2)</function>.
+ The <function>cs_get()</function> function will sometimes
+ (notably in the TCP/IP mode) read more than a single protocol package
+ off the network. When this happens, the extra package is stored
+ by the subsystem. After calling <function>cs_get()</function>, and before
+ waiting for more input, You should always call
+ <function>cs_more()</function>
+ to check if there's a full protocol package already read. If
+ <function>cs_more()</function>
+ returns 1,
+ <function>cs_get()</function>
+ can be used to immediately fetch the new package. For the
+ mOSI
+ subsystem, the function should always return 0, but if you want your
+ stuff to be protocol independent, you should use it.
+ </para>
+ <note>
+ <para>
+ The <function>cs_more()</function>
+ function is required because the RFC1729-method
+ does not provide a way of separating individual PDUs, short of
+ partially decoding the BER. Some other implementations will carefully
+ nibble at the packet by calling
+ <function>read(2)</function>
+ several times. This was felt to be too inefficient (or at least
+ clumsy) - hence the call for this extra function.
+ </para>
+ </note>
+ <synopsis>
+ int cs_look(COMSTACK handle);
+ </synopsis>
+ <para>
+ This function is useful when you're operating in nonblocking
+ mode. Call it when
+ <function>select(2)</function>
+ tells you there's something happening on the line. It returns one of
+ the following values:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term>CS_NONE</term>
+ <listitem><para>
+ No event is pending. The data found on the line was not a
+ complete package.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_CONNECT</term>
+ <listitem><para>
+ A response to your connect request has been received. Call
+ <function>cs_rcvconnect</function>
+ to process the event and to finalize the connection establishment.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_DISCON</term>
+ <listitem><para>
+ The other side has closed the connection (or maybe sent a disconnect
+ request - but do we care? Maybe later). Call
+ <function>cs_close</function> to close your end of the association
+ as well.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_LISTEN</term>
+ <listitem><para>
+ A connect request has been received.
+ Call <function>cs_listen</function> to process the event.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_DATA</term>
+ <listitem><para>
+ There's data to be found on the line.
+ Call <function>cs_get</function> to get it.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ <note>
+ <para>
+ You should be aware that even if
+ <function>cs_look()</function>
+ tells you that there's an event event pending, the corresponding
+ function may still return and tell you there was nothing to be found.
+ This means that only part of a package was available for reading. The
+ same event will show up again, when more data has arrived.
+ </para>
+ </note>
+ <synopsis>
+ int cs_fileno(COMSTACK h);
+ </synopsis>
+ <para>
+ Returns the file descriptor of the association. Use this when
+ file-level operations on the endpoint are required
+ (<function>select(2)</function> operations, specifically).
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="comstack.client">
+ <title>Client Side</title>
+ <synopsis>
+ int cs_connect(COMSTACK handle, void *address);
+ </synopsis>
+ <para>
+ Initiate a connection with the target at <literal>address</literal>
+ (more on addresses below). The function will return 0 on success, and 1 if
+ the operation does not complete immediately (this will only
+ happen on a nonblocking endpoint). In this case, use
+ <function>cs_rcvconnect</function> to complete the operation,
+ when <function>select(2)</function> or <function>poll(2)</function>
+ reports input pending on the association.
+ </para>
+ <synopsis>
+ int cs_rcvconnect(COMSTACK handle);
+ </synopsis>
+ <para>
+ Complete a connect operation initiated by <function>cs_connect()</function>.
+ It will return 0 on success; 1 if the operation has not yet completed (in
+ this case, call the function again later); -1 if an error has occurred.
+ </para>
+ </sect1>
+ <sect1 id="comstack.server">
+ <title>Server Side</title>
+ <para>
+ To establish a server under the <application>inetd</application>
+ server, you can use
+ </para>
+ <synopsis>
+ COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
+ int protocol);
+ </synopsis>
+ <para>
+ The <literal>socket</literal> parameter is an established socket (when
+ your application is invoked from <application>inetd</application>, the
+ socket will typically be 0.
+ The following parameters are identical to the ones for
+ <function>cs_create</function>.
+ </para>
+ <synopsis>
+ int cs_bind(COMSTACK handle, void *address, int mode)
+ </synopsis>
+ <para>
+ Binds a local address to the endpoint. Read about addresses below. The
+ <literal>mode</literal> parameter should be either
+ <literal>CS_CLIENT</literal> or <literal>CS_SERVER</literal>.
+ </para>
+ <synopsis>
+ int cs_listen(COMSTACK handle, char *addr, int *addrlen);
+ </synopsis>
+ <para>
+ Call this to process incoming events on an endpoint that has been
+ bound in listening mode. It will return 0 to indicate that the connect
+ request has been received, 1 to signal a partial reception, and -1 to
+ indicate an error condition.
+ </para>
+ <synopsis>
+ COMSTACK cs_accept(COMSTACK handle);
+ </synopsis>
+ <para>
+ This finalizes the server-side association establishment, after
+ cs_listen has completed successfully. It returns a new connection
+ endpoint, which represents the new association. The application will
+ typically wish to fork off a process to handle the association at this
+ point, and continue listen for new connections on the old
+ <literal>handle</literal>.
+ </para>
+ <para>
+ You can use the call
+ </para>
+ <synopsis>
+ const char *cs_addrstr(COMSTACK);
+ </synopsis>
+ <para>
+ on an established connection to retrieve the host-name of the remote host.
+ </para>
+ <note>
+ <para>
+ You may need to use this function with some care if your
+ name server service is slow or unreliable
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="comstack.addresses">
+ <title>Addresses</title>
+ <para>
+ The low-level format of the addresses are different depending on the
+ mode of communication you have chosen. A function is provided by each
+ of the lower layers to map a user-friendly string-form address to the
+ binary form required by the lower layers.
+ </para>
+ <synopsis>
+ void *cs_straddr(COMSTACK handle, const char *str);
+ </synopsis>
+ <para>
+ The format for TCP/IP and SSL addresses is:
+ </para>
+ <synopsis>
+ <host> [ ':' <portnum> ]
+ </synopsis>
+ <para>
+ The <literal>hostname</literal> can be either a domain name or an
+ IP address. The port number, if omitted, defaults to 210.
+ </para>
+ <para>
+ For TCP/IP and SSL, the special hostnames <literal>@</literal>,
+ maps to <literal>IN6ADDR_ANY_INIT</literal> with
+ IPV4 binding as well (bindv6only=0),
+ The special hostname <literal>@4</literal> binds to
+ <literal>INADDR_ANY</literal> (IPV4 only listener).
+ The special hostname <literal>@6</literal> binds to
+ <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
+ </para>
+ <para>
+ For UNIX sockets, the format of an address is the socket filename.
+ </para>
+ <para>
+ When a connection has been established, you can use
+ </para>
+ <synopsis>
+ const char *cs_addrstr(COMSTACK h);
+ </synopsis>
+ <para>
+ to retrieve the host name of the peer system. The function returns
+ a pointer to a static area, which is overwritten on the next call
+ to the function.
+ </para>
+ <para>
+ A fairly recent addition to the &comstack; module is the utility
+ function
+ </para>
+ <synopsis>
+ COMSTACK cs_create_host (const char *str, int blocking, void **vp);
+ </synopsis>
+ <para>
+ which is just a wrapper for <function>cs_create</function> and
+ <function>cs_straddr</function>. The <parameter>str</parameter>
+ is similar to that described for <function>cs_straddr</function>
+ but with a prefix denoting the &comstack; type. Prefixes supported
+ are <literal>tcp:</literal>, <literal>unix:</literal> and
+ <literal>ssl:</literal> for TCP/IP, UNIX and SSL respectively.
+ If no prefix is given, then TCP/IP is used.
+ The <parameter>blocking</parameter> is passed to
+ function <function>cs_create</function>. The third parameter
+ <parameter>vp</parameter> is a pointer to &comstack; stack type
+ specific values.
+ Parameter <parameter>vp</parameter> is reserved for future use.
+ Set it to <literal>NULL</literal>.
+ </para>
+ </sect1>
+ <sect1 id="comstack.ssl">
+ <title>SSL</title>
+ <para>
+ <synopsis>
+ void *cs_get_ssl(COMSTACK cs);
+ </synopsis>
+ Returns the SSL handle, <literal>SSL *</literal> for comstack. If comstack
+ is not of type SSL, NULL is returned.
+ </para>
+ <para>
+ <synopsis>
+ int cs_set_ssl_ctx(COMSTACK cs, void *ctx);
+ </synopsis>
+ Sets SSL context for comstack. The parameter is expected to be of type
+ <literal>SSL_CTX *</literal>. This function should be called just
+ after comstack has been created (before connect, bind, etc).
+ This function returns 1 for success; 0 for failure.
+ </para>
+ <para>
+ <synopsis>
+ int cs_set_ssl_certificate_file(COMSTACK cs, const char *fname);
+ </synopsis>
+ Sets SSL certificate for comstack as a PEM file. This function
+ returns 1 for success; 0 for failure.
+ </para>
+ <para>
+ <synopsis>
+ int cs_get_ssl_peer_certificate_x509(COMSTACK cs, char **buf, int *len);
+ </synopsis>
+ This function returns the peer certificate. If successful,
+ <literal>*buf</literal> and <literal>*len</literal> holds
+ X509 buffer and length respectively. Buffer should be freed
+ with <literal>xfree</literal>. This function returns 1 for success;
+ 0 for failure.
+ </para>
+ </sect1>
+ <sect1 id="comstack.diagnostics">
+ <title>Diagnostics</title>
+ <para>
+ All functions return -1 if an error occurs. Typically, the functions
+ will return 0 on success, but the data exchange functions
+ (<function>cs_get</function>, <function>cs_put</function>,
+ <function>cs_more</function>) follow special rules. Consult their
+ descriptions.
+ </para>
+ <para>
+ The error code for the COMSTACK can be retrieved using C macro
+ <function>cs_errno</function> which will return one
+ of the error codes <literal>CSYSERR</literal>,
+ <literal>CSOUTSTATE</literal>,
+ <literal>CSNODATA</literal>, ...
+ </para>
+ <synopsis>
+ int cs_errno(COMSTACK handle);
+ </synopsis>
+ <para>
+ You can the textual representation of the error code
+ by using <function>cs_errmsg</function> - which
+ works like <function>strerror(3)</function>
+ </para>
+ <synopsis>
+ const char *cs_errmsg(int n);
+ </synopsis>
+ <para>
+ It is also possible to get straight to the textual represenataion
+ without the error code by using
+ <function>cs_strerror</function>.
+ </para>
+ <synopsis>
+ const char *cs_strerror(COMSTACK h);
+ </synopsis>
+ </sect1>
+ <sect1 id="comstack.summary">
+ <title>Summary and Synopsis</title>
+ <synopsis><![CDATA[
+ #include <yaz/comstack.h>
+
+ #include <yaz/tcpip.h> /* this is for TCP/IP and SSL support */
+ #include <yaz/unix.h> /* this is for UNIX socket support */
+
+ COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
+
+ COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
+ int protocol);
+ COMSTACK cs_create_host(const char *str, int blocking,
+ void **vp);
+
+ int cs_bind(COMSTACK handle, int mode);
+
+ int cs_connect(COMSTACK handle, void *address);
+
+ int cs_rcvconnect(COMSTACK handle);
+
+ int cs_listen(COMSTACK handle);
+
+ COMSTACK cs_accept(COMSTACK handle);
+
+ int cs_put(COMSTACK handle, char *buf, int len);
+
+ int cs_get(COMSTACK handle, char **buf, int *size);
+
+ int cs_more(COMSTACK handle);
+
+ void cs_close(COMSTACK handle);
+
+ int cs_look(COMSTACK handle);
+
+ void *cs_straddr(COMSTACK handle, const char *str);
+
+ const char *cs_addrstr(COMSTACK h);
+]]>
+ </synopsis>
+ </sect1>
+ </chapter>
+ <chapter id="future">
+ <title>Future Directions</title>
+ <para>
+ We have a new and better version of the front-end server on the drawing
+ board. Resources and external commitments will govern when we'll be
+ able to do something real with it. Features should include greater
+ flexibility, greater support for access/resource control, and easy
+ support for Explain (possibly with Zebra as an extra database engine).
+ </para>
+ <para>
+ &yaz; is a BER toolkit and as such should support all protocols
+ out there based on that. We'd like to see running ILL applications.
+ It shouldn't be that hard. Another thing that would be interesting is
+ LDAP. Maybe a generic framework for doing IR using both LDAP and
+ Z39.50 transparently.
+ </para>
+ <para>
+ The SOAP implementation is incomplete. In the future we hope
+ to add more features to it. Perhaps make a WSDL/XML Schema compiler.
+ The authors of libxml2 are already working on XML Schema / RelaxNG
+ compilers so this may not be too hard.
+ </para>
+ <para>
+ It would be neat to have a proper module mechanism for the Generic
+ Frontend Server so that backend would be dynamically
+ loaded (as shared objects / DLLs).
+ </para>
+ <para>
+ Other than that, &yaz; generally moves in the directions which appear to
+ make the most people happy (including ourselves, as prime users of the
+ software). If there's something you'd like to see in here, then drop
+ us a note and let's see what we can come up with.
+ </para>
+ </chapter>
+ <reference id="reference">
+ <title>Reference</title>
+ <partintro id="reference-introduction">
+ <para>
+ The material in this chapter is drawn directly from the individual
+ manual entries.
+ </para>
+ </partintro>
+ &manref;
+ </reference>
+ <appendix id="list-oids">
+ <title>List of Object Identifiers</title>
+ <para>
+ These is a list of object identifiers that are built into YAZ.
+ </para>
+ &std-oid-table;
+ </appendix>
+ <appendix id="bib1-diagnostics">
+ <title>Bib-1 diagnostics</title>
+ <para>
+ List of Bib-1 diagnostics that are known to YAZ.
+ </para>
+ &bib1-diag-table;
+ </appendix>
+ <appendix id="sru-diagnostics">
+ <title>SRU diagnostics</title>
+ <para>
+ List of SRU diagnostics that are known to YAZ.
+ </para>
+ &srw-diag-table;
+ </appendix>
+ <appendix id="license">
+ <title>License</title>
+ <sect1 id="license.indexdata">
+ <title>Index Data Copyright</title>
+ <para>
+ Copyright © ©right-year; Index Data.
+ </para>
+ <para>
+ All rights reserved.
+ </para>
+ <para>
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions are met:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Neither the name of Index Data nor the names of its contributors
+ may be used to endorse or promote products derived from this
+ software without specific prior written permission.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ THIS SOFTWARE IS PROVIDED BY INDEX DATA ``AS IS'' AND ANY
+ EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL INDEX DATA BE LIABLE FOR
+ ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ SUCH DAMAGE.
+ </para>
+ </sect1>
+ </appendix>
+ <appendix id="indexdata">
+ <title>About Index Data</title>
+ <para>
+ Index Data is a consulting and software-development enterprise that
+ specializes in library and information management systems. Our
+ interests and expertise span a broad range of related fields, and one
+ of our primary, long-term objectives is the development of a powerful
+ information management
+ system with open network interfaces and hyper-media capabilities.
+ </para><para>
+ We make this software available free of charge, on a fairly unrestrictive
+ license; as a service to the networking community, and to further the
+ development of quality software for open network communication.
+ </para><para>
+ We'll be happy to answer questions about the software, and about ourselves
+ in general.
+ </para>
+ <para>
+ <address>
+ Index Data ApS
+ <street>Amagerfælledvej 56</street>
+ <postcode>2300 Copenhagen S</postcode>
+ <country>Denmark</country>
+ Email <email>info@indexdata.dk</email>
+ </address>
+ </para>
+ <para>
+ The Hacker's Jargon File has the following to say about the
+ use of the
+ prefix "YA" in the name of a software product.
+ </para>
+ <para>
+ <citation>
+ Yet Another. adj. 1. Of your own work: A
+ humorous allusion often used in titles to acknowledge that the
+ topic is not original, though the content is. As in "Yet Another
+ AI Group" or "Yet Another Simulated Annealing Algorithm".
+ 2. Of
+ others' work: Describes something of which there are already far
+ too many.
+ </citation>
+ </para>
+ </appendix>
+ <appendix id="credits">
+ <title>Credits</title>
+ <para>
+ This appendix lists individuals that have contributed in the development
+ of &yaz;. Some have contributed with code, while others have provided bug
+ fixes or suggestions. If we're missing somebody, of if you, for
+ whatever reason, don't like to be listed here, let us know.
+ </para>
+ <itemizedlist>
+ <listitem><para>
+ Gary Anderson
+ </para></listitem>
+ <listitem><para>
+ Dimitrios Andreadis
+ </para></listitem>
+ <listitem><para>
+ Morten Bøgeskov
+ </para></listitem>
+ <listitem><para>
+ Rocco Carbone
+ </para></listitem>
+ <listitem><para>
+ Matthew Carey
+ </para></listitem>
+ <listitem><para>
+ Hans van Dalen
+ </para></listitem>
+ <listitem><para>
+ Irina Dijour
+ </para></listitem>
+ <listitem><para>
+ Larry E. Dixson
+ </para></listitem>
+ <listitem><para>
+ Hans van den Dool
+ </para></listitem>
+ <listitem><para>
+ Mads Bondo Dydensborg
+ </para></listitem>
+ <listitem><para>
+ Franck Falcoz
+ </para></listitem>
+ <listitem><para>
+ Kevin Gamiel
+ </para></listitem>
+ <listitem><para>
+ Morten Garkier Hendriksen
+ </para></listitem>
+ <listitem><para>
+ Morten Holmqvist
+ </para></listitem>
+ <listitem><para>
+ Ian Ibbotson
+ </para></listitem>
+ <listitem><para>
+ Shigeru Ishida
+ </para></listitem>
+ <listitem><para>
+ Heiko Jansen
+ </para></listitem>
+ <listitem><para>
+ David Johnson
+ </para></listitem>
+ <listitem><para>
+ Oleg Kolobov
+ </para></listitem>
+ <listitem><para>
+ Giannis Kosmas
+ </para></listitem>
+ <listitem><para>
+ Kang-Jin Lee
+ </para></listitem>
+ <listitem><para>
+ Pieter Van Lierop
+ </para></listitem>
+ <listitem><para>
+ Stefan Lohrum
+ </para></listitem>
+ <listitem><para>
+ Ronald van der Meer
+ </para></listitem>
+ <listitem><para>
+ Thomas W. Place
+ </para></listitem>
+ <listitem><para>
+ Peter Popovics
+ </para></listitem>
+ <listitem><para>
+ Jacob Chr. Poulsen
+ </para></listitem>
+ <listitem><para>
+ Ko van der Sloot
+ </para></listitem>
+ <listitem><para>
+ Mike Taylor
+ </para></listitem>
+ <listitem><para>
+ Rustam T. Usmanov
+ </para></listitem>
+ <listitem><para>
+ Charles Woodfield
+ </para></listitem>
+ <listitem><para>
+ Tom André Øverland
+ </para></listitem>
+ </itemizedlist>
+ </appendix>
+</book>
+
+<!-- Keep this comment at the end of the file
+Local variables:
+mode: nxml
+nxml-child-indent: 1
+End:
+-->