merged, consolidated and upgraded into DocBook source.
+++ /dev/null
-$Id: README,v 1.2 2006-01-17 10:35:12 mike Exp $
-
-This directory contains documentation generated by human hands. In
-contrast, ../dox contains documentation automatically generated by
-doxygen from the API declarations.
-
-Eventually, everything in this directory will become an integrated YP2
-manual written in DocBook, like the YAZ and Zebra manuals. For now,
-though, we have separate files, which might perhaps best be read in
-the specified order:
-
-README -- this file.
-filters -- lists and briefly describes the various filters.
-config -- describes the YP2 configuration file.
-multidb -- virtual databases and multi-db searching (complicated!)
-classes -- describes some of the classes in the YP2 source code.
-
+++ /dev/null
-$Id: classes,v 1.4 2006-01-17 09:26:44 mike Exp $
-
-
-A Hitch-Hiker's Guide to the YP2 Classes
-========================================
-
-Introductory Notes
-------------------
-
-In general, classes seem to be named big-endianly, so that
-"FactoryFilter" is not a filter for factories, but a factory that
-produces filters, and "FactoryStatic" is a factory for the statically
-registered filters (as opposed to those that are dynamically loaded).
-
-
-Classes
-=======
-
-yp::FactoryFilter
------------------
-
-A factory class that exists primarily to provide the create() method,
-which takes the name of a filter class as its argument and returns a
-new filter of that type. To enable this, the factory must first be
-populated by calling add_creator() for static filters (this is done by
-the FactoryStatic class, see below) and add_creator_dyn() for filters
-loaded dynamically.
-
-yp2::FactoryStatic
-------------------
-
-A subclass of FactoryFilter which is responsible for registering all
-the statically defined filter types. It does this by knowing about
-all those filters' structures, which are listed in its constructor.
-Merely instantiating this class registers all the static classes. It
-is for the benefit of this class that struct yp2_filter_struct exists,
-and that all the filter classes provide a static object of that type.
-
-yp2::filter::Base
------------------
-
-The virtual base class of all filters. The filter API is, on the
-surface at least, extremely simple: two methods. configure() is
-passed a DOM tree representing that part of the configuration file
-that pertains to this filter instance, and is expected to walk that
-tree extracting relevant information. And process() processes a
-Package (see below). That surface simplicitly is a bit misleading, as
-process() needs to know a lot about the Package class in order to do
-anything useful.
-
-yp2::filter::AuthSimple, Backend_test, FrontendNet, etc.
---------------------------------------------------------
-
-Individual filters. Each of these is implemented by a header and a
-source file, named filter_*.hpp and filter_*.cpp respectively. All
-the header files should be pretty much identical, in that they declare
-the class, including a private Rep class and a member pointer to it,
-and the two public methods. The only extra information in any filter
-header is additional private types and members (the latter should
-really all be in the Rep anyway).
-
-The source file for each filter needs to supply:
-- a definition of the private Rep class
-- some boilerplate constructors and destructors
-- a configure() method that uses the appropriate XML fragment
-- most important, the prcoess() method that does all the actual work.
-
-In a better world, each of the filters (nine so far) would be
-documented separately. Very, very briefly, though:
-- auth_simple: authentication against a plain-text file
-- backend_test: trivial ztest-like dummy hardwired back-end
-- frontend_net: receives incoming Z39.50 and SRW PSUs
-- log: writes log messages
-- multi: fans out searches to multiple back-ends
-- session_shared: global sharing of cached result-sets
-- template: NOT TO BE USED: just a template source file for new filters
-- virt_db: chooses which server/db to use based on database name
-- z3950_client: forwards PDUs to a nominated Z39.50 server
-
-### What is "filter_dl.cpp"? It's to do with dynamic loading, but in
-what way and to what extent does it function as a filter?
-
-See the file "filters" for more information.
-
-
-yp2::Package
-------------
-
-Represents a package on its way through the series of filters that
-make up a route. This is essentially a Z39.50 or SRU APDU together
-with information about where it came from, which is modified as it
-passes through the various filters.
-
-
-yp2::Pipe
----------
-
-This class provides a compatibility layer so that we have an IPC
-mechanism that works the same under Unix and Windows. It's not
-particularly exciting.
-
-
-yp2::Router and yp2::RouterChain
---------------------------------
-
-###
-
-
-yp2::RouterFleXML
------------------
-
-###
-
-
-yp2::Session
-------------
-
-###
-
-yp2::ThreadPoolSocketObserver
------------------------------
-
-###
-
-yp2::util
----------
-
-A namespace of various small utility functions and classes, collected
-together for convenience. Most importantly, includes the
-yp2::util::odr class, a wrapper for YAZ's ODR facilities.
-
-
-yp2::xml
----------
-
-A namespace of various XML utility functions and classes, collected
-together for convenience.
-
-
-Other Source Files
-==================
-
-ex_router_flexml.cpp -- the main function of the proxy.
-yp2_prog.cpp -- identical to the above: it's not clear why.
-test_*.cpp -- unit-tests for various modules.
+++ /dev/null
-$Id: config,v 1.3 2006-01-18 10:29:08 mike Exp $
-
-The Guerilla Guide to Configuring YP2
-=====================================
-
-Introduction: invocation, schema and namespace
-----------------------------------------------
-
-YP2 configuration is by means of a single XML file, the name of which
-is supplied as the sole command-line argument to the "yp2" binary.
-This document briefly describes the format of that configuration file.
-
-An XML schema, config.xsd, is provided for validating configuration
-files. It can be used by (among other tools) the "xmllint" program
-supplied as part of the libxml2 distribution:
-
- xmllint --noout --schema config.xsd config1.xml
-
-(A recent version of libxml2 is required, as support for XML Schemas
-is a relatively recent addition.)
-
-All elements and attributes are in the namespace
- http://indexdata.dk/yp2/config/1
-This is most easily achieved by setting the default namespace on the
-top-level element, as here:
-
- <yp2 xmlns="http://indexdata.dk/yp2/config/1">
-
-
-Overview of XML structure
--------------------------
-
-The top-level element is <yp2>. This contains a <start> element, a
-<filters> element and a <routes> element, in that order. <filters> is
-optional; the other two are mandatory. All three are non-repeatable.
-
-<start> is empty, but carries a "route" attribute, whose value is the
-name of route at which to start running -- a bit like the name of the
-start production in a grammar.
-
-If present, <filters> contains zero or more <filter> elements; filters
-carry a "type" attribute and contain various elements that provide
-suitable configuration for filters of that type. The filter-specific
-elements are described below. Filters defined in this part of the
-file must carry an "id" attribute so that they can be referenced from
-elsewhere.
-
-<routes> contains zero or more <route> elements, each of which must
-carry an "id" element. One of the routes must have the id that was
-specified for the start route in the <start> element's "route"
-attribute. Each route contains zero or more <filter> elements. These
-are of two types. They may be empty, but carry a "refid" attribute
-whose value is the same as the "id" of a filter previously defined in
-the <filters> section. Alternatively, a route within a filter may
-omit the "refid" attribute, but contain configuration elements similar
-to those used for filters defined in the <filters> section.
-
-
-Filter configuration
---------------------
-
-All <filter> elements have in common that they must carry a "type"
-attribute whose value is one of the supported ones, listed in the
-schema file and discussed below. In additional, <filters>s occurring
-the <filters> section must have an "id" attribute, and those occurring
-within a route must have either a "refid" attribute referencing a
-previously defined filter or contain its own configuration
-information.
-
-In general, each filter recognises different configuration elements
-within its element, as each filter has different functionality. These
-are as follows:
-
-<filter type="frontend_net">
- <threads>10</threads>
- <port>@:9000</port>
-</filter>
-
-<filter type="z3950_client">
- <timeout>30</timeout>
-</filter>
-
-<filter type="log">
- <message>B</message>
-</filter>
-
-<filter type="virt_db">
- <virtual>
- <database>loc</database>
- <target>z3950.loc.gov:7090/voyager</target>
- </virtual>
- <virtual>
- <database>idgils</database>
- <target>indexdata.dk/gils</target>
- </virtual>
-</filter>
-
-<filter type="auth_simple">
- <userRegister>../etc/example.simple-auth</userRegister>
-</filter>
-
-<filter type="backend_test"/>
-
-<filter type="multi">
- ### Not yet resolved
-</filter>
-
-<filter type="session_shared">
- ### Not yet defined
-</filter>
-
-<filter type="template"/>
-
+++ /dev/null
-$Id: filters,v 1.1 2006-01-17 10:43:14 mike Exp $
-
-
-YP2 Filter Classes for Superhuman Geniuses
-==========================================
-
-Introductory Notes
-------------------
-
-It's useful to think of a YP2 configuration file as a program (written
-in XML, but that's an implementation detail) providing a small number
-of primitives and operations, but operating on a very complex data
-type, namely the Package. A package represents a Z39.50 or SRW/U
-request (whether for Init, Search, Scan, etc.) together with
-information about where it came from. Packages come from front-end
-filters such as frontend_end, which reads them from the network; other
-front-end filters are possible. They then pass along a route
-consisting of a sequence of filters, each of which transforms the
-package and may also have side-effects such as generating logging.
-Eventually, the route will yield a response, which is sent back to the
-origin.
-
-There are many kinds of filter: some that are defined statically as
-part of YP2, and other that may be provided by third parties and
-dynamically loaded. They all conform to the same simple API of
-essentially two methods: configure() is passed a DOM tree representing
-that part of the configuration file that pertains to this filter
-instance, and is expected to walk that tree extracting relevant
-information; and process() processes a Package.
-
-While all filters provide the same API, there are different modes of
-functionality. Some filters are sources -- they create packages
-(frontend_net); others are sinks -- they consume packages and return a
-result (z3950_client, backend_test); the others are true filters, that
-read process and pass on the packages they are fed (auth_simple, log,
-multi, session_shared, template, virt_db).
-
-
-Filters
--------
-
-The filters are here named by the string that is used as the "type"
-attribute of a <filter> element in the configuration file to request
-them, with the class that implements them in parentheses.
-
-auth_simple (yp2::filter::AuthSimple)
-
- Simple authentication and authorisation. The configuration
- specifies the name of a file that is the user register, which
- lists username:password pairs, one per line, colon separated.
- When a session begins, it is rejected unless username and
- passsword are supplied, and match a pair in the register.
-
-backend_test (yp2::filter::Backend_test)
-
- A sink that provides dummy responses in the manner of the
- "yaz-ztest" Z39.50 server. This is useful only for testing.
-
-frontend_net (yp2::filter::FrontendNet)
-
- A source that accepts Z39.50 and SRW connections from a port
- specified in the configuration, reads protocol units, and
- feeds them into the next filter, eventually returning the
- result to the origin.
-
-log (yp2::filter::Log)
-
- Writes logging information to standard output, and passes on
- the package unchanged.
-
-multi (yp2::filter::Multi)
-
- Performs multicast searching. See the extended discussion
- in the file "multidb".
-
-session_shared (yp2::filter::SessionShared)
-
- When this is finished, it will implement global sharing of
- result sets (i.e. between threads and therefore between
- clients), but it's not yet done.
-
-template (yp2::filter::Template)
-
- Does nothing at all, merely passing the packet on. (Maybe it
- should be called "nop" or "passthrough"?) This exists not to
- be used, but to be copied -- to become the skeleton of new
- filters as they are written.
-
-virt_db (yp2::filter::Virt_db)
-
- Performs virtual database selection. See the extended
- discussion in the file "multidb".
-
-z3950_client (yp2::filter::Z3950Client)
-
- Performs Z39.50 searching and retrieval by proxying the
- packages that are passed to it. Init requests are sent to the
- address specified in the VAL_PROXY otherInfo attached to the
- request: this may have been specified by client, or generated
- by a virt_db filter earlier in the route. Subsequent requests
- are sent to the same address, which is remembered at Init time
- in a Session object.
-
-
-Directions
-----------
-
-Some other filters that do not yet exist but would be useful:
-frontend_cli (source) - command-line interface for generating requests.
-srw2z3950 (filter) - translate SRW requests into Z39.50 requests.
-srw_client (sink) - performs SRW searching and retrieval.
-sru_client (sink) - performs SRU searching and retrieval.
-opensearch_client (sink) - A9 OpenSearch searching and retrieval.
-
-
+++ /dev/null
-$Id: multidb,v 1.1 2006-01-17 11:10:30 mike Exp $
-
-
-YP2 Virtual Databases and Multicast Searching
-=============================================
-
-Two of YP2's filters are concerned with multiple-database operations.
-Of these, "virt_db" can work alone to control the routing of searches
-to one of a number of servers, while "multi" can work with the output
-of "virt_db" to perform multicast searching, merging the results into
-a unified result-set. The interaction between these two filters is
-necessarily complex, reflecting the real complexity of multicast
-searching in a protocol such as Z39.50 that separates initialisation
-from searching, with the database to search known only during the
-latter operation.
-
-### Much, much more to say!
-
projects / metaproxy-moved-to-github.git / commitdiff