X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=doc%2Fzoom.xml;h=6c6199f7039834f9bbac66101812e8fa8cfc161a;hb=01a40d09e6a956b3faa2743c62db290c252fa4bb;hp=0a5700a2d3ec93e7e9bd706067de84747827e61c;hpb=fa4910b8543b112f2afc2d92b4f959bce7943b5f;p=metaproxy-moved-to-github.git diff --git a/doc/zoom.xml b/doc/zoom.xml index 0a5700a..6c6199f 100644 --- a/doc/zoom.xml +++ b/doc/zoom.xml @@ -1,4 +1,4 @@ - @@ -7,7 +7,7 @@ Metaproxy - Index Data + Index Data @@ -15,24 +15,25 @@ 3mp Metaproxy Module - + zoom Metaproxy ZOOM Module - - DESCRIPTION + + + DESCRIPTION This filter implements a generic client based on ZOOM of YAZ. The client implements the protocols that ZOOM C does: Z39.50, SRU - (GET, POST, SOAP) and SOLR . + (GET, POST, SOAP) and Solr . This filter only deals with Z39.50 on input. The following services are supported: init, search, present and close. The backend target - is selected based on the database as part of this search and + is selected based on the database as part search and not as part of init. @@ -52,263 +53,868 @@ - CONFIGURATION - - The configuration consists of three parts: torus, - fieldmap and cclmap. - + + CONFIGURATION + + The configuration consists of six parts: torus, + fieldmap, cclmap, + contentProxy, log + and zoom. + + + torus - The torus element specifies target profiles - and takes the following content: + The torus element specifies target profiles + and takes the following content: - attribute url - URL of Web service to be used to fetch target profile - for a given database (udb). The special sequence - %db of the URL is replaced by the - actual database specified as part of Search. - - - - attribute xsldir - Directory that is searched for XSL stylesheets. Stylesheets - are specified in the target profile by the - transform element. - - - - element records - Local target profiles. This element may includes zero or - more record elements (one per target - profile). See section TARGET PROFILE. - - - + + attribute url + + + URL of Web service to be used when fetch target profiles from + a remote service (Torus normally). + + + The sequence %query is replaced with a CQL + query for the Torus search. + + + The special sequence %realm is replaced by value + of attribute realm or by realm DATABASE argument. + + + The special sequence %db is replaced with + a single database while searching. Note that this sequence + is no longer needed, because the %query can already + query for a single database by using CQL query + udb==.... + + + + + attribute content_url + + + URL of Web service to be used to fetch target profile + for a given database (udb) of type content. Semantics otherwise like + url attribute above. + + + + + attribute auth_url + + + URL of Web service to be used to for auth/IP lookup. If this + defined, all access is granted or denied as part of Z39.50 Init + by the ZOOM module and the use of database parameters realm and + torus_url is not allowed. If this setting is not defined, + all access is allowed and realm and/or torus_url may be used. + + + + + attribute auth_hostname + + + Limits IP lookup to a given logical hostname. + + + + + attribute realm + + + The default realm value. Used for %realm in URL, unless + specified in DATABASE parameter. + + + + + attribute proxy + + + HTTP proxy to bse used for fetching target profiles. + + + + + attribute xsldir + + + Directory that is searched for XSL stylesheets. Stylesheets + are specified in the target profile by the + transform element. + + + + + attribute element_transform + + + Specifies the element that triggers retrieval and transform using + the parameters elementSet, recordEncoding, requestSyntax, transform + from the target profile. Default value + is "pz2", due to the fact that for historical reasons the + common format is that used in Pazpar2. + + + + + attribute element_raw + + + Specifies an element that triggers retrieval using the + parameters elementSet, recordEncoding, requestSyntax from the + target profile. Same actions as for element_transform, but without + the XSL transform. Useful for debugging. + The default value is "raw". + + + + + attribute explain_xsl + + + Specifies a stylesheet that converts one or more Torus records + to ZeeExplain records. The content of recordData is assumed to be + holding each Explain record. + + + + + attribute record_xsl + + + Specifies a stylesheet that converts retrieval records after + transform/literal operations. + + + When Metaproxy creates a content proxy session, the XSL parameter + cproxyhost is passed to the transform. + + + + + element records + + + Local target profiles. This element may includes zero or + more record elements (one per target + profile). See section TARGET PROFILE. + + + + + + fieldmap - The fieldmap may be specified zero or more times and - specifies the map from CQL fields to CCL fields and takes the - following content: + The fieldmap may be specified zero or more times and + specifies the map from CQL fields to CCL fields and takes the + following content: - attribute cql - CQL field. - - - - attribute ccl - CCL field. - - - + + attribute cql + + + CQL field that we are mapping "from". + + + + + attribute ccl + + + CCL field that we are mapping "to". + + + + + + cclmap - The final part of the configuration consists of zero or more - cclmap elements that specifies a base - CCL profile to be used for all targets. This configuration, thus, will - be combined with cclmap-definitions from the target profile. - - - QUERIES - - The ZOOM filter accepts three query types: RPN(Type-1), CCL and - CQL. - - - Queries are converted in two separate steps. In the first step - the input query is converted to RPN/Type-1. This is always - the common internal format between step 1 and step 2. - In step 2 the query is converted to the native query type of the target. + The third part of the configuration consists of zero or more + cclmap elements that specifies + base CCL profile to be used for all targets. + This configuration, thus, will be combined with cclmap-definitions + from the target profile. + + + contentProxy - Step 1: for RPN, the query is passed unmodified to the target. - - - Step 1: for CCL, the query is converted to RPN via - cclmap elements part of the target profile. - - - Step 1: For CQL, the query is converted to CCL. The mappings of - CQL fields to CCL fields are handled by fieldmap - elements as part of the target profile. The resulting query, CCL, - is the converted to RPN using the schema mentioned earlier (via - cclmap). + The contentProxy element controls content proxy'in. + This section + is optional and must only be defined if content proxy'ing is enabled. + + + attribute config_file + + + Specifies the file that configures the cf-proxy system. Metaproxy + uses setting sessiondir and + proxyhostname from that file to configure + name of proxy host and directory of parameter files for the cf-proxy. + + + + + attribute server + + + Specifies the content proxy host. The host is of the form + host[:port]. That is without a method (such as HTTP) and optional + port number. + + + + This setting is deprecated. Use the config_file (above) + to inform about the proxy server. + + + + + + attribute tmp_file + + + Specifies a filename of a session file for content proxy'ing. The + file should be an absolute filename that includes + XXXXXX which is replaced by a unique filename + using the mkstemp(3) system call. The default value of this + setting is /tmp/cf.XXXXXX.p. + + + + This setting is deprecated. Use the config_file (above) + to inform about the session file area. + + + + + + + + log - Step 2: If the target is Z39.50-based, it is passed verbatim (RPN). - If the target is SRU-based, the RPN will be converted to CQL. - If the target is SOLR-based, the RPN will be converted to SOLR's query - type. + The log element controls logging for the + ZOOM filter. - + + + attribute apdu + + + If the value of apdu is "true", then protocol packages + (APDUs and HTTP packages) from the ZOOM filter will be + logged to the yaz_log system. A value of "false" will + not perform logging of protocol packages (the default + behavior). + + + + + - TARGET PROFILE + + zoom - The following elements are honored by the ZOOM module of Metaproxy. - Note that unknown elements are silently ignored. There are several - elements in use that makes no sense to the ZOOM module. + The zoom element controls settings for the + ZOOM. - - authentication - - Authentication parameters to be sent to the target. For - Z39.50 targets, this will be sent as part of the - Init Request. - - - If this value is omitted or empty, not authentication information - is simply omitted. - - - - - - piggyback - - A value of 1/true is a hint to the ZOOM module that this Z39.50 - target supports piggyback searches, ie Search Response with - records. Any other value (false) will prevent the ZOOM module - to make use of piggyback (all records part of Present Response). - - - - - - queryEncoding - - If this value is defined, all queries will be converted - to this encoding. This should be used for all Z39.50 targets that - do not use UTF-8 for query terms. - - - - - - udb - - This value is required and specifies the unique database for - this profile . All target profiles should hold a unique database. - - - - - - cclmap_* - - This value specifies CCL field (qualifier) definition for some - field. For Z39.50 targets this most likely will specify the - mapping to a numeric use attribute + a structure attribute. - For SRU targets, the use attribute should be string based, in - order to make the RPN to CQL conversion work properly (step 2). - - - - - - elementSet - - Specifies the elementSet to be sent to the target if record - transform is enabled (not to be used with the record_transform - module). The record transform is enabled only if the client uses - record syntax = XML and element set = - pz2. If record transform is not enabled, this - setting is not used and the element set specified by the client - is passed verbatim. - - - - - - recordEncoding - - Specifies the character encoding of records that are returned - by the target. This is primarily used for targets were records - are not UTF-8 encoded already. This setting is only used - if the record transform is enabled (see description of elementSet). - - - - - - requestSyntax - - Specifies the record syntax to be specified for the target - if record transform is enabled; see description of elementSet. - If record transform is not enabled, the record syntax of the - client is passed verbatim to the target. - - - - - - sru - - If this setting is set, it specifies that the target is web service - based and must be one of : get, - post, soap - or solr. - - - - - - transform - - Specifies a XSL stylesheet filename to be used if record - transform is anabled; see desciprion of elementSet. - - - - - - zurl - - This is setting is mandatory and specifies the ZURL of the - targetin the form of host/database. - - - + + attribute timeout + + + Is an integer that specifies, in seconds, how long an operation + may take before ZOOM gives up. Default value is 40. + + + + + attribute proxy_timeout + + + Is an integer that specifies, in seconds, how long an operation + a proxy check will wait before giving up. Default value is 1. + + + + + + + + QUERY HANDLING + + The ZOOM filter accepts three query types: RPN(Type-1), CCL and + CQL. + + + Queries are converted in two separate steps. In the first step + the input query is converted to RPN/Type-1. This is always + the common internal format between step 1 and step 2. + In step 2 the query is converted to the native query type of the target. + + + Step 1: for RPN, the query is passed un-modified to the target. + + + Step 1: for CCL, the query is converted to RPN via + cclmap + elements part of + the target profile as well as + base CCL maps. + + + Step 1: For CQL, the query is converted to CCL. The mappings of + CQL fields to CCL fields are handled by + fieldmap + elements as part of the target profile. The resulting query, CCL, + is the converted to RPN using the schema mentioned earlier (via + cclmap). + + + Step 2: If the target is Z39.50-based, it is passed verbatim (RPN). + If the target is SRU-based, the RPN will be converted to CQL. + If the target is Solr-based, the RPN will be converted to Solr's query + type. + + + + + SORTING + + The ZOOM module actively handle CQL sorting - using the SORTBY parameter + which was introduced in SRU version 1.2. The conversion from SORTBY clause + to native sort for some target is driven by the two parameters: + + sortStrategy + + and + sortmap_field + . + + + If a sort field that does not have an equivalent + sortmap_-mapping is passed un-modified through the + conversion. It doesn't throw a diagnostic. + + + + + TARGET PROFILE + + The ZOOM module is driven by a number of settings that specifies how + to handle each target. + Note that unknown elements are silently ignored. + + + The elements, in alphabetical order, are: + + + + authentication + + Authentication parameters to be sent to the target. For + Z39.50 targets, this will be sent as part of the + Init Request. Authentication consists of two components: username + and password, separated by a slash. + + + If this value is omitted or empty no authentication information is sent. + + + + + + authenticationMode + + Specifies how authentication parameters are passed to server + for SRU. Possible values are: url + and basic. For the url mode username and password + are carried in URL arguments x-username and x-password. + For the basic mode, HTTP basic authentication is used. + The settings only takes effect + if authentication + is set. + + + If this value is omitted HTTP basic authencation is used. + + + + + + cclmap_field + + This value specifies CCL field (qualifier) definition for some + field. For Z39.50 targets this most likely will specify the + mapping to a numeric use attribute + a structure attribute. + For SRU targets, the use attribute should be string based, in + order to make the RPN to CQL conversion work properly (step 2). + + + + + + cfAuth + + When cfAuth is defined, its value will be used as authentication + to backend target and authentication setting will be specified + as part of a database. This is like a "proxy" for authentication and + is used for Connector Framework based targets. + + + + + + cfProxy + + Specifies HTTP proxy for the target in the form + host:port. + + + + + + cfSubDB + + Specifies sub database for a Connector Framework based target. + + + + + + contentAuthentication + + Specifies authentication info to be passed to a content connector. + This is only used if content-user and content-password are omitted. + + + + + + contentConnector + + Specifies a database for content-based proxy'ing. + + + + + + elementSet + + Specifies the elementSet to be sent to the target if record + transform is enabled (not to be confused' with the record_transform + module). The record transform is enabled only if the client uses + record syntax = XML and a element set determined by + the element_transform / + element_raw from the configuration. + By default that is the element sets pz2 + and raw. + If record transform is not enabled, this setting is + not used and the element set specified by the client + is passed verbatim. + + + + + + literalTransform + + Specifies a XSL stylesheet to be used if record + transform is anabled; see description of elementSet. + The XSL transform is only used if the element set is set to the + value of element_transform in the configuration. + + + The value of literalTransform is the XSL - string encoded. + + + + + + piggyback + + A value of 1/true is a hint to the ZOOM module that this Z39.50 + target supports piggyback searches, ie Search Response with + records. Any other value (false) will prevent the ZOOM module + to make use of piggyback (all records part of Present Response). + + + + + + queryEncoding + + If this value is defined, all queries will be converted + to this encoding. This should be used for all Z39.50 targets that + do not use UTF-8 for query terms. + + + + + + recordEncoding + + Specifies the character encoding of records that are returned + by the target. This is primarily used for targets were records + are not UTF-8 encoded already. This setting is only used + if the record transform is enabled (see description of elementSet). + + + + + + requestSyntax + + Specifies the record syntax to be specified for the target + if record transform is enabled; see description of elementSet. + If record transform is not enabled, the record syntax of the + client is passed verbatim to the target. + + + + + + sortmap_field + + This value the native field for a target. The form of the value is + given by sortStrategy. + + + + + + sortStrategy + + Specifies sort strategy for a target. One of: + z3950, type7, + cql, sru11 or + embed. The embed chooses type-7 + or CQL sortby depending on whether Type-1 or CQL is + actually sent to the target. + + + + + + sru + + If this setting is set, it specifies that the target is web service + based and must be one of : get, + post, soap + or solr. + + + + + + sruVersion + + Specifies the SRU version to use. It unset, version 1.2 will be + used. Some servers do not support this version, in which case + version 1.1 or even 1.0 could be set it. + + + + + + transform + + Specifies a XSL stylesheet filename to be used if record + transform is anabled; see description of elementSet. + The XSL transform is only used if the element set is set to the + value of element_transform in the configuration. + + + + + + udb + + This value is required and specifies the unique database for + this profile . All target profiles should hold a unique database. + + + + + + urlRecipe + + The value of this field is a string that generates a dynamic link + based on record content. If the resulting string is non-zero in length + a new field, metadata with attribute + type="generated-url" is generated. + The contents of this field is the result of the URL recipe conversion. + The urlRecipe value may refer to an existing metadata element by + ${field[pattern/result/flags]}, which will take content + of field and perform a regular expression conversion using the pattern + given. For example: ${md-title[\s+/+/g]} takes + metadata element title and converts one or more + spaces to a plus character. + + + + + + zurl + + This is setting is mandatory and specifies the ZURL of the + target in the form of host/database. The HTTP method should + not be provided as this is guessed from the "sru" attribute value. + + + + + + + DATABASE parameters + + Extra information may be carried in the Z39.50 Database or SRU path, + such as authentication to be passed to backend etc. Some of + the parameters override TARGET profile values. The format is + + + udb,parm1=value1&parm2=value2&... + + + Where udb is the unique database recognised by the backend and parm1, + value1, .. are parameters to be passed. The following describes the + supported parameters. Like form values in HTTP the parameters and + values are URL encoded. The separator, though, between udb and parameters + is a comma rather than a question mark. What follows question mark are + HTTP arguments (in this case SRU arguments). + + + The database parameters, in alphabetical order, are: + + + + content-password + + + The password to be used for content proxy session. If this parameter + is not given, value of parameter password is passed + to content proxy session. + + + + + content-proxy + + + Specifies proxy to be used for content proxy session. If this parameter + is not given, value of parameter proxy is passed + to content proxy session. + + + + + content-user + + + The user to be used for content proxy session. If this parameter + is not given, value of parameter user is passed + to content proxy session. + + + + + cproxysession + + + Specifies the session ID for content proxy. This parameter is, generally, + not used by anything but the content proxy itself when invoking + Metaproxy via SRU. + + + + + nocproxy + + + If this parameter is specified, content-proxying is disabled + for the search. + + + + + password + + + Specifies password to be passed to backend. It is also passed + to content proxy session unless overriden by content-password. + If this parameter is omitted, the password will be taken from + TARGET profile setting + + authentication + + . + + + + + proxy + + + Specifies one or more proxies for backend. If this parameter is + omitted, the proxy will be taken from TARGET profile setting + + cfProxy. + The parameter is a list of comma-separated host:port entries. + Bost host and port must be given for each proxy. + + + + + realm + + + Session realm to be used for this target, changed the resulting + URL to be used for getting a target profile, by changing the + value that gets substituted for the %realm string. This parameter + is not allowed if access is controlled by + auth_url + in configuration. + + + + + retry + + + Optional paramater. If the value is 0, retry on failure is + disabled for the ZOOM module. Any other value enables retry + on failure. If this parameter is omitted, then the value of + retryOnFailure + from the Torus record is used (same values). + + + + + + torus_url + + + Sets the URL to be used for Torus records fetch - overriding value + of url attribute of element torus + in zoom configuration. This parameter is not allowed if access is + controlled by + auth_url in configuration. + + + + + + user + + + Specifies user to be passed to backend. It is also passed + to content proxy session unless overriden by content-user. + If this parameter is omitted, the user will be taken from TARGET + profile setting + + authentication + + . + + + + + x-parm + + + All parameters that has prefix x, dash are passed verbatim + to the backend. + + + + - SCHEMA - - + + SCHEMA + + - - EXAMPLES + + + EXAMPLES - The following configuration illustrates most of the - facilities: + In example below Target definitions (Torus records) are fetched + from a web service via a proxy. A CQL profile is configured which + maps to a set of CCL fields ("no field", au, tu and su). Presumably + the target definitions fetched maps the CCL to their native RPN. + A CCL "ocn" is mapped for all targets. Logging of APDUs are enabled, + and a timeout is given. - + + + ]]> - - - SEE ALSO + + Here is another example with two locally defined targets: A + Solr target and a Z39.50 target. + + + + + + ocs-test + t=z + u=title t=z + solr + ocs-test.indexdata.com/solr/select + + + loc + t=l,r + u=4 t=l,r + lx2.loc.gov:210/LCDB_MARC8 + + + + + + +]]> + + + + + + SEE ALSO metaproxy @@ -322,21 +928,13 @@ - + ©right;