X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=doc%2Fzoom.xml;h=3f2849a4167284b3f112131d5319ea5b065ab8c4;hb=39fe3f90ff2cf99ec39053df3c0835ab87b0deba;hp=c6a09e942cceb033e88995dca9a7d54a07095d08;hpb=eb2f36fbec12a2063183d5ae4a9568c55e28acdc;p=metaproxy-moved-to-github.git
diff --git a/doc/zoom.xml b/doc/zoom.xml
index c6a09e9..3f2849a 100644
--- a/doc/zoom.xml
+++ b/doc/zoom.xml
@@ -1,4 +1,4 @@
-
@@ -15,18 +15,19 @@
3mp
Metaproxy Module
-
+
zoom
Metaproxy ZOOM Module
-
- DESCRIPTION
+
+
+ DESCRIPTION
This filter implements a generic client based on
ZOOM of YAZ.
The client implements the protocols that ZOOM C does: Z39.50, SRU
- (GET, POST, SOAP) and SOLR .
+ (GET, POST, SOAP) and Solr .
@@ -52,323 +53,805 @@
- CONFIGURATION
-
- The configuration consists of four parts: torus,
- fieldmap, cclmap
- and log.
-
- torus
-
- The torus element specifies target profiles
- and takes the following content:
-
-
-
- attribute url
-
-
- URL of Web service to be used to fetch target profile
- for a given database (udb). The special sequence
- %db of the URL is replaced by the
- actual database specified as part of Search.
-
-
-
-
- attribute proxy
-
-
- HTTP proxy to bse used for fetching target profiles.
-
-
-
-
- attribute xsldir
-
-
- Directory that is searched for XSL stylesheets. Stylesheets
- are specified in the target profile by the
- transform element.
-
-
-
-
- attribute element_transform
-
-
- Specifies the element that triggers retrieval and transform using
- the parameters elementSet, recordEncoding, requestSyntax, transform
- from the target profile. Default value
- is "pz2", due to the fact that for historical reasons the
- common format is that used in Pazpar2.
-
-
-
-
- attribute element_raw
-
-
- Specifies an element that triggers retrieval using the
- parameters elementSet, recordEncoding, requestSyntax from the
- target profile. Same actions as for element_transform, but without
- the XSL transform. Useful for debugging.
- The default value is "raw".
-
-
-
-
- element records
-
-
- Local target profiles. This element may includes zero or
- more record elements (one per target
- profile). See section TARGET PROFILE.
-
-
-
-
-
- fieldmap
-
- The fieldmap may be specified zero or more times and
- specifies the map from CQL fields to CCL fields and takes the
- following content:
-
-
-
- attribute cql
-
-
- CQL field that we are mapping "from".
-
-
-
-
- attribute ccl
-
-
- CCL field that we are mapping "to".
-
-
-
-
-
- cclmap
-
- The third part of the configuration consists of zero or more
- cclmap elements that specifies
- base CCL profile to be used for all targets.
- This configuration, thus, will be combined with cclmap-definitions
- from the target profile.
-
-
- log
-
- The log element controls logging for the
- ZOOM filter.
-
-
-
- attribute apdu
-
-
- If the value of apdu is "true", then protocol packages
- (APDUs and HTTP packages) from the ZOOM filter will be
- logged to the yaz_log system. A value of "false" will
- not perform logging of protocol packages (the default
- behavior).
-
-
-
-
-
-
- QUERY HANDLING
-
- The ZOOM filter accepts three query types: RPN(Type-1), CCL and
- CQL.
-
+
+ CONFIGURATION
+
+ The configuration consists of six parts: torus,
+ fieldmap, cclmap,
+ contentProxy, log
+ and zoom.
+
+
+ torus
- Queries are converted in two separate steps. In the first step
- the input query is converted to RPN/Type-1. This is always
- the common internal format between step 1 and step 2.
- In step 2 the query is converted to the native query type of the target.
+ The torus element specifies target profiles
+ and takes the following content:
+
+
+ attribute url
+
+
+ URL of Web service to be used when fetch target profiles from
+ a remote service (Torus normally).
+
+
+ The sequence %query is replaced with a CQL
+ query for the Torus search.
+
+
+ The special sequence %realm is replaced by value
+ of attribute realm or by realm DATABASE argument.
+
+
+ The special sequence %db is replaced with
+ a single database while searching. Note that this sequence
+ is no longer needed, because the %query can already
+ query for a single database by using CQL query
+ udb==....
+
+
+
+
+ attribute content_url
+
+
+ URL of Web service to be used to fetch target profile
+ for a given database (udb) of type content. Semantics otherwise like
+ url attribute above.
+
+
+
+
+ attribute auth_url
+
+
+ URL of Web service to be used to for auth/IP lookup. If this
+ defined, all access is granted or denied as part of Z39.50 Init
+ by the ZOOM module and the use of database parameters realm and
+ torus_url is not allowed. If this setting is not defined,
+ all access is allowed and realm and/or torus_url may be used.
+
+
+
+
+ attribute auth_hostname
+
+
+ Limits IP lookup to a given logical hostname.
+
+
+
+
+ attribute realm
+
+
+ The default realm value. Used for %realm in URL, unless
+ specified in DATABASE parameter.
+
+
+
+
+ attribute proxy
+
+
+ HTTP proxy to bse used for fetching target profiles.
+
+
+
+
+ attribute xsldir
+
+
+ Directory that is searched for XSL stylesheets. Stylesheets
+ are specified in the target profile by the
+ transform element.
+
+
+
+
+ attribute element_transform
+
+
+ Specifies the element that triggers retrieval and transform using
+ the parameters elementSet, recordEncoding, requestSyntax, transform
+ from the target profile. Default value
+ is "pz2", due to the fact that for historical reasons the
+ common format is that used in Pazpar2.
+
+
+
+
+ attribute element_raw
+
+
+ Specifies an element that triggers retrieval using the
+ parameters elementSet, recordEncoding, requestSyntax from the
+ target profile. Same actions as for element_transform, but without
+ the XSL transform. Useful for debugging.
+ The default value is "raw".
+
+
+
+
+ attribute explain_xsl
+
+
+ Specifies a stylesheet that converts one or more Torus records
+ to ZeeExplain records. The content of recordData is assumed to be
+ holding each Explain record.
+
+
+
+
+ attribute record_xsl
+
+
+ Specifies a stylesheet that converts retrieval records after
+ transform/literal operations.
+
+
+ When Metaproxy creates a content proxy session, the XSL parameter
+ cproxyhost is passed to the transform.
+
+
+
+
+ element records
+
+
+ Local target profiles. This element may includes zero or
+ more record elements (one per target
+ profile). See section TARGET PROFILE.
+
+
+
+
+
+
+ fieldmap
- Step 1: for RPN, the query is passed unmodified to the target.
+ The fieldmap may be specified zero or more times and
+ specifies the map from CQL fields to CCL fields and takes the
+ following content:
+
+
+ attribute cql
+
+
+ CQL field that we are mapping "from".
+
+
+
+
+ attribute ccl
+
+
+ CCL field that we are mapping "to".
+
+
+
+
+
+
+ cclmap
- Step 1: for CCL, the query is converted to RPN via
- cclmap elements part of the target profile.
+ The third part of the configuration consists of zero or more
+ cclmap elements that specifies
+ base CCL profile to be used for all targets.
+ This configuration, thus, will be combined with cclmap-definitions
+ from the target profile.
+
+
+ contentProxy
- Step 1: For CQL, the query is converted to CCL. The mappings of
- CQL fields to CCL fields are handled by fieldmap
- elements as part of the target profile. The resulting query, CCL,
- is the converted to RPN using the schema mentioned earlier (via
- cclmap).
+ The contentProxy element controls content proxy'in.
+ This section
+ is optional and must only be defined if content proxy'ing is enabled.
+
+
+ attribute config_file
+
+
+ Specifies the file that configures the cf-proxy system. Metaproxy
+ uses setting sessiondir and
+ proxyhostname from that file to configure
+ name of proxy host and directory of parameter files for the cf-proxy.
+
+
+
+
+ attribute server
+
+
+ Specifies the content proxy host. The host is of the form
+ host[:port]. That is without a method (such as HTTP) and optional
+ port number.
+
+
+
+ This setting is deprecated. Use the config_file (above)
+ to inform about the proxy server.
+
+
+
+
+
+ attribute tmp_file
+
+
+ Specifies a filename of a session file for content proxy'ing. The
+ file should be an absolute filename that includes
+ XXXXXX which is replaced by a unique filename
+ using the mkstemp(3) system call. The default value of this
+ setting is /tmp/cf.XXXXXX.p.
+
+
+
+ This setting is deprecated. Use the config_file (above)
+ to inform about the session file area.
+
+
+
+
+
+
+
+ log
- Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
- If the target is SRU-based, the RPN will be converted to CQL.
- If the target is SOLR-based, the RPN will be converted to SOLR's query
- type.
+ The log element controls logging for the
+ ZOOM filter.
-
+
+
+ attribute apdu
+
+
+ If the value of apdu is "true", then protocol packages
+ (APDUs and HTTP packages) from the ZOOM filter will be
+ logged to the yaz_log system. A value of "false" will
+ not perform logging of protocol packages (the default
+ behavior).
+
+
+
+
+
- TARGET PROFILE
+
+ zoom
- The following elements are honored by the ZOOM module of Metaproxy.
- Note that unknown elements are silently ignored. There are several
- elements in use that makes no sense to the ZOOM module.
+ The zoom element controls settings for the
+ ZOOM.
-
- authentication
-
- Authentication parameters to be sent to the target. For
- Z39.50 targets, this will be sent as part of the
- Init Request.
-
-
- If this value is omitted or empty, not authentication information
- is simply omitted.
-
-
-
-
-
- piggyback
-
- A value of 1/true is a hint to the ZOOM module that this Z39.50
- target supports piggyback searches, ie Search Response with
- records. Any other value (false) will prevent the ZOOM module
- to make use of piggyback (all records part of Present Response).
-
-
-
-
-
- queryEncoding
-
- If this value is defined, all queries will be converted
- to this encoding. This should be used for all Z39.50 targets that
- do not use UTF-8 for query terms.
-
-
-
-
-
- udb
-
- This value is required and specifies the unique database for
- this profile . All target profiles should hold a unique database.
-
-
-
-
-
- cclmap_*
-
- This value specifies CCL field (qualifier) definition for some
- field. For Z39.50 targets this most likely will specify the
- mapping to a numeric use attribute + a structure attribute.
- For SRU targets, the use attribute should be string based, in
- order to make the RPN to CQL conversion work properly (step 2).
-
-
-
-
-
- elementSet
-
- Specifies the elementSet to be sent to the target if record
- transform is enabled (not to be confused' with the record_transform
- module). The record transform is enabled only if the client uses
- record syntax = XML and a element set determined by
- the element_transform /
- element_raw from the configuration.
- By default that is the element sets pz2
- and raw.
- If record transform is not enabled, this setting is
- not used and the element set specified by the client
- is passed verbatim.
-
-
-
-
-
- recordEncoding
-
- Specifies the character encoding of records that are returned
- by the target. This is primarily used for targets were records
- are not UTF-8 encoded already. This setting is only used
- if the record transform is enabled (see description of elementSet).
-
-
-
-
-
- requestSyntax
-
- Specifies the record syntax to be specified for the target
- if record transform is enabled; see description of elementSet.
- If record transform is not enabled, the record syntax of the
- client is passed verbatim to the target.
-
-
-
-
-
- sru
-
- If this setting is set, it specifies that the target is web service
- based and must be one of : get,
- post, soap
- or solr.
-
-
-
-
-
- transform
-
- Specifies a XSL stylesheet filename to be used if record
- transform is anabled; see desciprion of elementSet.
- The XSL transform is only used if the element set is set to the
- value of element_transform in the configuration.
-
-
-
-
-
- zurl
-
- This is setting is mandatory and specifies the ZURL of the
- target in the form of host/database. The HTTP method should
- not be provide as this is guessed from the "sru" attribute value.
-
-
-
+
+ attribute timeout
+
+
+ Is an integer that specifies, in seconds, how long an operation
+ may take before ZOOM gives up. Default value is 40.
+
+
+
+
+ attribute proxy_timeout
+
+
+ Is an integer that specifies, in seconds, how long an operation
+ a proxy check will wait before giving up. Default value is 1.
+
+
+
+
+
- SCHEMA
-
-
+
+ QUERY HANDLING
+
+ The ZOOM filter accepts three query types: RPN(Type-1), CCL and
+ CQL.
+
+
+ Queries are converted in two separate steps. In the first step
+ the input query is converted to RPN/Type-1. This is always
+ the common internal format between step 1 and step 2.
+ In step 2 the query is converted to the native query type of the target.
+
+
+ Step 1: for RPN, the query is passed un-modified to the target.
+
+
+ Step 1: for CCL, the query is converted to RPN via
+ cclmap
+ elements part of
+ the target profile as well as
+ base CCL maps.
+
+
+ Step 1: For CQL, the query is converted to CCL. The mappings of
+ CQL fields to CCL fields are handled by
+ fieldmap
+ elements as part of the target profile. The resulting query, CCL,
+ is the converted to RPN using the schema mentioned earlier (via
+ cclmap).
+
+
+ Step 2: If the target is Z39.50-based, it is passed verbatim (RPN).
+ If the target is SRU-based, the RPN will be converted to CQL.
+ If the target is Solr-based, the RPN will be converted to Solr's query
+ type.
+
-
- EXAMPLES
+
+
+ SORTING
+
+ The ZOOM module actively handle CQL sorting - using the SORTBY parameter
+ which was introduced in SRU version 1.2. The conversion from SORTBY clause
+ to native sort for some target is driven by the two parameters:
+
+ sortStrategy
+
+ and
+ sortmap_field
+ .
+
- The following configuration illustrates most of the
- facilities:
+ If a sort field that does not have an equivalent
+ sortmap_-mapping is passed un-modified through the
+ conversion. It doesn't throw a diagnostic.
+
+
+
+
+ TARGET PROFILE
+
+ The ZOOM module is driven by a number of settings that specifies how
+ to handle each target.
+ Note that unknown elements are silently ignored.
+
+
+ The elements, in alphabetical order, are:
+
+
+
+ authentication
+
+ Authentication parameters to be sent to the target. For
+ Z39.50 targets, this will be sent as part of the
+ Init Request. Authentication consists of two components: username
+ and password, separated by a slash.
+
+
+ If this value is omitted or empty no authentication information is sent.
+
+
+
+
+
+ authenticationMode
+
+ Specifies how authentication parameters are passed to server
+ for SRU. Possible values are: url
+ and basic. For the url mode username and password
+ are carried in URL arguments x-username and x-password.
+ For the basic mode, HTTP basic authentication is used.
+ The settings only takes effect
+ if authentication
+ is set.
+
+
+ If this value is omitted HTTP basic authencation is used.
+
+
+
+
+
+ cclmap_field
+
+ This value specifies CCL field (qualifier) definition for some
+ field. For Z39.50 targets this most likely will specify the
+ mapping to a numeric use attribute + a structure attribute.
+ For SRU targets, the use attribute should be string based, in
+ order to make the RPN to CQL conversion work properly (step 2).
+
+
+
+
+
+ cfAuth
+
+ When cfAuth is defined, its value will be used as authentication
+ to backend target and authentication setting will be specified
+ as part of a database. This is like a "proxy" for authentication and
+ is used for Connector Framework based targets.
+
+
+
+
+
+ cfProxy
+
+ Specifies HTTP proxy for the target in the form
+ host:port.
+
+
+
+
+
+ cfSubDB
+
+ Specifies sub database for a Connector Framework based target.
+
+
+
+
+
+ contentConnector
+
+ Specifies a database for content-based proxy'ing.
+
+
+
+
+
+ elementSet
+
+ Specifies the elementSet to be sent to the target if record
+ transform is enabled (not to be confused' with the record_transform
+ module). The record transform is enabled only if the client uses
+ record syntax = XML and a element set determined by
+ the element_transform /
+ element_raw from the configuration.
+ By default that is the element sets pz2
+ and raw.
+ If record transform is not enabled, this setting is
+ not used and the element set specified by the client
+ is passed verbatim.
+
+
+
+
+
+ literalTransform
+
+ Specifies a XSL stylesheet to be used if record
+ transform is anabled; see description of elementSet.
+ The XSL transform is only used if the element set is set to the
+ value of element_transform in the configuration.
+
+
+ The value of literalTransform is the XSL - string encoded.
+
+
+
+
+
+ piggyback
+
+ A value of 1/true is a hint to the ZOOM module that this Z39.50
+ target supports piggyback searches, ie Search Response with
+ records. Any other value (false) will prevent the ZOOM module
+ to make use of piggyback (all records part of Present Response).
+
+
+
+
+
+ queryEncoding
+
+ If this value is defined, all queries will be converted
+ to this encoding. This should be used for all Z39.50 targets that
+ do not use UTF-8 for query terms.
+
+
+
+
+
+ recordEncoding
+
+ Specifies the character encoding of records that are returned
+ by the target. This is primarily used for targets were records
+ are not UTF-8 encoded already. This setting is only used
+ if the record transform is enabled (see description of elementSet).
+
+
+
+
+
+ requestSyntax
+
+ Specifies the record syntax to be specified for the target
+ if record transform is enabled; see description of elementSet.
+ If record transform is not enabled, the record syntax of the
+ client is passed verbatim to the target.
+
+
+
+
+
+ sortmap_field
+
+ This value the native field for a target. The form of the value is
+ given by sortStrategy.
+
+
+
+
+
+ sortStrategy
+
+ Specifies sort strategy for a target. One of:
+ z3950, type7,
+ cql, sru11 or
+ embed. The embed chooses type-7
+ or CQL sortby depending on whether Type-1 or CQL is
+ actually sent to the target.
+
+
+
+
+
+ sru
+
+ If this setting is set, it specifies that the target is web service
+ based and must be one of : get,
+ post, soap
+ or solr.
+
+
+
+
+
+ sruVersion
+
+ Specifies the SRU version to use. It unset, version 1.2 will be
+ used. Some servers do not support this version, in which case
+ version 1.1 or even 1.0 could be set it.
+
+
+
+
+
+ transform
+
+ Specifies a XSL stylesheet filename to be used if record
+ transform is anabled; see description of elementSet.
+ The XSL transform is only used if the element set is set to the
+ value of element_transform in the configuration.
+
+
+
+
+
+ udb
+
+ This value is required and specifies the unique database for
+ this profile . All target profiles should hold a unique database.
+
+
+
+
+
+ urlRecipe
+
+ The value of this field is a string that generates a dynamic link
+ based on record content. If the resulting string is non-zero in length
+ a new field, metadata with attribute
+ type="generated-url" is generated.
+ The contents of this field is the result of the URL recipe conversion.
+ The urlRecipe value may refer to an existing metadata element by
+ ${field[pattern/result/flags]}, which will take content
+ of field and perform a regular expression conversion using the pattern
+ given. For example: ${md-title[\s+/+/g]} takes
+ metadata element title and converts one or more
+ spaces to a plus character.
+
+
+
+
+
+ zurl
+
+ This is setting is mandatory and specifies the ZURL of the
+ target in the form of host/database. The HTTP method should
+ not be provided as this is guessed from the "sru" attribute value.
+
+
+
+
+
+
+ DATABASE parameters
+
+ Extra information may be carried in the Z39.50 Database or SRU path,
+ such as authentication to be passed to backend etc. Some of
+ the parameters override TARGET profile values. The format is
+
+
+ udb,parm1=value1&parm2=value2&...
+
+
+ Where udb is the unique database recognised by the backend and parm1,
+ value1, .. are parameters to be passed. The following describes the
+ supported parameters. Like form values in HTTP the parameters and
+ values are URL encoded. The separator, though, between udb and parameters
+ is a comma rather than a question mark. What follows question mark are
+ HTTP arguments (in this case SRU arguments).
+
+
+ The database parameters, in alphabetical order, are:
+
+
+
+ content-password
+
+
+ The password to be used for content proxy session. If this parameter
+ is not given, value of parameter password is passed
+ to content proxy session.
+
+
+
+
+ content-proxy
+
+
+ Specifies proxy to be used for content proxy session. If this parameter
+ is not given, value of parameter proxy is passed
+ to content proxy session.
+
+
+
+
+ content-user
+
+
+ The user to be used for content proxy session. If this parameter
+ is not given, value of parameter user is passed
+ to content proxy session.
+
+
+
+
+ cproxysession
+
+
+ Specifies the session ID for content proxy. This parameter is, generally,
+ not used by anything but the content proxy itself when invoking
+ Metaproxy via SRU.
+
+
+
+
+ nocproxy
+
+
+ If this parameter is specified, content-proxying is disabled
+ for the search.
+
+
+
+
+ password
+
+
+ Specifies password to be passed to backend. It is also passed
+ to content proxy session unless overriden by content-password.
+ If this parameter is omitted, the password will be taken from
+ TARGET profile setting
+
+ authentication
+
+ .
+
+
+
+
+ proxy
+
+
+ Specifies one or more proxies for backend. If this parameter is
+ omitted, the proxy will be taken from TARGET profile setting
+
+ cfProxy.
+ The parameter is a list of comma-separated host:port entries.
+ Bost host and port must be given for each proxy.
+
+
+
+
+ realm
+
+
+ Session realm to be used for this target, changed the resulting
+ URL to be used for getting a target profile, by changing the
+ value that gets substituted for the %realm string. This parameter
+ is not allowed if access is controlled by
+ auth_url
+ in configuration.
+
+
+
+
+ retry
+
+
+ Optional paramater. If the value is 0, retry on failure is
+ disabled for the ZOOM module. Any other value enables retry
+ on failure. If this parameter is omitted, then the value of
+ retryOnFailure
+ from the Torus record is used (same values).
+
+
+
+
+
+ torus_url
+
+
+ Sets the URL to be used for Torus records fetch - overriding value
+ of url attribute of element torus
+ in zoom configuration. This parameter is not allowed if access is
+ controlled by
+ auth_url in configuration.
+
+
+
+
+
+ user
+
+
+ Specifies user to be passed to backend. It is also passed
+ to content proxy session unless overriden by content-user.
+ If this parameter is omitted, the user will be taken from TARGET
+ profile setting
+
+ authentication
+
+ .
+
+
+
+
+ x-parm
+
+
+ All parameters that has prefix x, dash are passed verbatim
+ to the backend.
+
+
+
+
+
+
+ SCHEMA
+
+
+
+
+
+ EXAMPLES
+
+ In example below Target definitions (Torus records) are fetched
+ from a web service via a proxy. A CQL profile is configured which
+ maps to a set of CCL fields ("no field", au, tu and su). Presumably
+ the target definitions fetched maps the CCL to their native RPN.
+ A CCL "ocn" is mapped for all targets. Logging of APDUs are enabled,
+ and a timeout is given.
@@ -376,7 +859,7 @@
-
+
@@ -384,14 +867,45 @@
+
]]>
-
-
- SEE ALSO
+
+ Here is another example with two locally defined targets: A
+ Solr target and a Z39.50 target.
+
+
+
+
+
+ ocs-test
+ t=z
+ u=title t=z
+ solr
+ ocs-test.indexdata.com/solr/select
+
+
+ loc
+ t=l,r
+ u=4 t=l,r
+ z3950.loc.gov:7090/voyager
+
+
+
+
+
+
+]]>
+
+
+
+
+
+ SEE ALSO
metaproxy
@@ -405,21 +919,13 @@
-
+
©right;