X-Git-Url: http://git.indexdata.com/?p=yaz-moved-to-github.git;a=blobdiff_plain;f=doc%2Ftools.xml;h=3e711d0ee27a81e62588b2300012c7e4b8d4e88d;hp=9ca9e2470cd140096cfbc15ac00a24d5b5974e7a;hb=7a98e9bfbb9d5fe7d44822a9838e3becbdce9363;hpb=757c8c0ffb82023c9059d9a5997515f65ad68561 diff --git a/doc/tools.xml b/doc/tools.xml index 9ca9e24..3e711d0 100644 --- a/doc/tools.xml +++ b/doc/tools.xml @@ -1,4 +1,4 @@ - + Supporting Tools @@ -129,11 +129,11 @@ query ::= top-set query-struct. - top-set ::= [ '@attrset' string ] + top-set ::= [ '@attrset' string ] query-struct ::= attr-spec | simple | complex | '@term' term-type query - attr-spec ::= '@attr' [ string ] string query-struct + attr-spec ::= '@attr' [ string ] string query-struct complex ::= operator query-struct query-struct. @@ -452,7 +452,7 @@ The CCL parser obeys the following grammar for the FIND argument. The syntax is annotated by in the lines prefixed by - ‐‐. + --. @@ -825,7 +825,7 @@ @directive value - +
CCL directives @@ -890,7 +890,7 @@
- + CCL API All public definitions can be found in the header file @@ -1263,6 +1263,13 @@ int cql_transform_FILE(cql_transform_t ct, value the attribute value. + The character * (asterisk) has special meaning + when used in the RPN pattern. + Each occurrence of * is substituted with the + CQL matching name (index, relation, qualifier etc). + This facility can be used to copy a CQL name verbatim to the RPN result. + + The following CQL patterns are recognized: @@ -1283,6 +1290,11 @@ int cql_transform_FILE(cql_transform_t ct, http://www.loc.gov/zing/cql/cql-indexes/v1.0/. If this pattern is not defined, the mapping will fail. + + The pattern, + index.set.* + is used when no other index pattern is matched. + @@ -1385,28 +1397,41 @@ int cql_transform_FILE(cql_transform_t ct, + + + set + + + + This specification defines a default CQL context set for index names. + The value on the right hand side is the URI for the set. + + + + - CQL to RPN mapping file + + CQL to RPN mapping file This simple file defines two context sets, three indexes and three relations, a position pattern and a default structure. @@ -1441,6 +1466,49 @@ int cql_transform_FILE(cql_transform_t ct, + + CQL to RPN string attributes + + In this example we allow any index to be passed to RPN as + a use attribute. + + + + + The http://bogus/rpn context set is also the default + so we can make queries such as + + title = a + + which is converted to + + @attr 2=3 @attr 4=1 @attr 3=3 @attr 1=title "a" + + + + + CQL to RPN using Bath Profile + + The file etc/pqf.properties has mappings from + the Bath Profile and Dublin Core to RPN. + If YAZ is installed as a package it's usually located + in /usr/share/yaz/etc and part of the + development package, such as libyaz-dev. + + CQL to XCQL conversion @@ -1473,30 +1541,186 @@ void cql_to_xml_stdio(struct cql_node *cn, FILE *f); The basic YAZ representation of an OID is an array of integers, - terminated with the value -1. The &odr; module provides two - utility-functions to create and copy this type of data elements: + terminated with the value -1. This integer is of type + Odr_oid. - + + Fundamental OID operations and the type Odr_oid + are defined in yaz/oid_util.h. + + + An OID can either be declared as a automatic variable or it can + allocated using the memory utilities or ODR/NMEM. It's + guaranteed that an OID can fit in OID_SIZE integers. + + Create OID on stack + + We can create an OID for the Bib-1 attribute set with: + + Odr_oid bib1[OID_SIZE]; + bib1[0] = 1; + bib1[1] = 2; + bib1[2] = 840; + bib1[3] = 10003; + bib1[4] = 3; + bib1[5] = 1; + bib1[6] = -1; + + + + + And OID may also be filled from a string-based representation using + dots (.). This is achieved by function + + int oid_dotstring_to_oid(const char *name, Odr_oid *oid); + + This functions returns 0 if name could be converted; -1 otherwise. + + Using oid_oiddotstring_to_oid + + We can fill the Bib-1 attribute set OID easier with: + + Odr_oid bib1[OID_SIZE]; + oid_oiddotstring_to_oid("1.2.840.10003.3.1", bib1); + + + + + We can also allocate an OID dynamically on a ODR stream with: - Odr_oid *odr_getoidbystr(ODR o, char *str); + Odr_oid *odr_getoidbystr(ODR o, const char *str); + This creates an OID from string-based representation using dots. + This function take an &odr; stream as parameter. This stream is used to + allocate memory for the data elements, which is released on a + subsequent call to odr_reset() on that stream. + + + Using odr_getoidbystr + + We can create a OID for the Bib-1 attribute set with: + + Odr_oid *bib1 = odr_getoidbystr(odr, "1.2.840.10003.3.1"); + + + - Creates an OID based on a string-based representation using dots (.) - to separate elements in the OID. + The function + + char *oid_oid_to_dotstring(const Odr_oid *oid, char *oidbuf) + + does the reverse of oid_oiddotstring_to_oid. It + converts an OID to the string-based representation using dots. + The supplied char buffer oidbuf holds the resulting + string and must be at least OID_STR_MAX in size. - - Odr_oid *odr_oiddup(ODR odr, Odr_oid *o); - - - Creates a copy of the OID referenced by the o - parameter. - Both functions take an &odr; stream as parameter. This stream is used to - allocate memory for the data elements, which is released on a - subsequent call to odr_reset() on that stream. + OIDs can be copied with oid_oidcpy which takes + two OID lists as arguments. Alternativly, an OID copy can be allocated + on a ODR stream with: + + Odr_oid *odr_oiddup(ODR odr, const Odr_oid *o); + + + + + OIDs can be compared with oid_oidcmp which returns + zero if the two OIDs provided are identical; non-zero otherwise. + + OID database + + From YAZ version 3 and later, the oident system has been replaced + by an OID database. OID database is a misnomer .. the old odient + system was also a database. + + + The OID database is really just a map between named Object Identifiers + (string) and their OID raw equivalents. Most operations either + convert from string to OID or other way around. + + + Unfortunately, whenever we supply a string we must also specify the + OID class. The class is necessary because some + strings correspond to multiple OIDs. An example of such a string is + Bib-1 which may either be an attribute-set + or a diagnostic-set. + + + Applications using the YAZ database should include + yaz/oid_db.h. + + + A YAZ database handle is of type yaz_oid_db_t. + Actually that's a pointer. You need not think deal with that. + YAZ has a built-in database which can be considered "constant" for + most purposes. + We can get hold that by using function yaz_oid_std. + + + All functions with prefix yaz_string_to_oid + converts from class + string to OID. We have variants of this + operation due to different memory allocation strategies. + + + All functions with prefix + yaz_oid_to_string converts from OID to string + + class. + + + Create OID with YAZ DB + + We can create an OID for the Bib-1 attribute set on the ODR stream + odr with: + + Odr_oid *bib1 = + yaz_string_to_oid_odr(yaz_oid_std(), CLASS_ATTSET, "Bib-1", odr); + + This is more complex than using odr_getoidbystr. + You would only use yaz_string_to_oid_odr when the + string (here Bib-1) is supplied by a user or configuration. + + + + + Standard OIDs + + + All the object identifers in the standard OID database as returned + by yaz_oid_std can referenced directly in a + program as a constant OID. + Each constant OID is prefixed with yaz_oid_ - + followed by OID class (lowercase) - then by OID name (normalized and + lowercase). + + + See for list of all object identifiers + built into YAZ. + These are declared in yaz/oid_std.h but are + included by yaz/oid_db.h as well. + + + Use a built-in OID + + We can allocate our own OID filled with the constant OID for + Bib-1 with: + + Odr_oid *bib1 = odr_oiddup(o, yaz_oid_attset_bib1); + + + + + + OID oident + + + + The oident utility has been removed from YAZ version 3. This + sub section only applies to YAZ version 2. + + The OID module provides a higher-level representation of the @@ -1730,18 +1954,6 @@ typedef struct oident oidbuf and returning its address. - - Finally, the module provides the following utility functions, whose - meaning should be obvious: - - - - void oid_oidcpy(int *t, int *s); - void oid_oidcat(int *t, int *s); - int oid_oidcmp(int *o1, int *o2); - int oid_oidlen(int *o); - - The OID module has been criticized - and perhaps rightly so @@ -1758,8 +1970,8 @@ typedef struct oident + - Nibble Memory @@ -2030,7 +2242,7 @@ typedef struct oident - YAZ_MARC_MARXML + YAZ_MARC_MARCXML The resulting record is converted to MARCXML. @@ -2079,6 +2291,267 @@ typedef struct oident + + Retrieval Facility + + YAZ version 2.1.20 or later includes a Retrieval facility tool + which allows a SRU/Z39.50 to describe itself and perform record + conversions. The idea is the following: + + + + + An SRU/Z39.50 client sends a retrieval request which includes + a combination of the following parameters: syntax (format), + schema (or element set name). + + + + + + The retrieval facility is invoked with parameters in a + server/proxy. The retrieval facility matches the parameters a set of + "supported" retrieval types. + If there is no match, the retrieval signals an error + (syntax and / or schema not supported). + + + + + + For a successful match, the backend is invoked with the same + or altered retrieval parameters (syntax, schema). If + a record is received from the backend, it is converted to the + frontend name / syntax. + + + + + + The resulting record is sent back the client and tagged with + the frontend syntax / schema. + + + + + + + The Retrieval facility is driven by an XML configuration. The + configuration is neither Z39.50 ZeeRex or SRU ZeeRex. But it + should be easy to generate both of them from the XML configuration. + (unfortunately the two versions + of ZeeRex differ substantially in this regard). + + + Retrieval XML format + + All elements should be covered by namespace + http://indexdata.com/yaz . + The root element node must be retrievalinfo. + + + The retrievalinfo must include one or + more retrieval elements. Each + retrieval defines specific combination of + syntax, name and identifier supported by this retrieval service. + + + The retrieval element may include any of the + following attributes: + + syntax (REQUIRED) + + + Defines the record syntax. Possible values is any + of the names defined in YAZ' OID database or a raw + OID in (n.n ... n). + + + + name (OPTIONAL) + + + Defines the name of the retrieval format. This can be + any string. For SRU, the value, is equivalent to schema (short-hand); + for Z39.50 it's equivalent to simple element set name. + + + + identifier (OPTIONAL) + + + Defines the URI schema name of the retrieval format. This can be + any string. For SRU, the value, is equivalent to URI schema. + For Z39.50, there is no equivalent. + + + + + + + The retrieval may include one + backend element. If a backend + element is given, it specifies how the records are retrieved by + some backend and how the records are converted from the backend to + the "frontend". + + + The attributes, name and syntax + may be specified for the backend element. These + semantics of these attributes is equivalent to those for the + retrieval. However, these values are passed to + the "backend". + + + The backend element may includes one or more + conversion instructions (as children elements). The supported + conversions are: + + marc + + + The marc element specifies a conversion + to - and from ISO2709 encoded MARC and + &marcxml;/MarcXchange. + The following attributes may be specified: + + + inputformat (REQUIRED) + + + Format of input. Supported values are + marc (for ISO2709); and xml + for MARCXML/MarcXchange. + + + + + outputformat (REQUIRED) + + + Format of output. Supported values are + line (MARC line format); + marcxml (for MARCXML), + marc (ISO2709), + marcxhcange (for MarcXchange). + + + + + inputcharset (OPTIONAL) + + + Encoding of input. For XML input formats, this need not + be given, but for ISO2709 based inputformats, this should + be set to the encoding used. For MARC21 records, a common + inputcharset value would be marc-8. + + + + + outputcharset (OPTIONAL) + + + Encoding of output. If outputformat is XML based, it is + strongly recommened to use utf-8. + + + + + + + + + xslt + + + The xslt element specifies a conversion + via &xslt;. The following attributes may be specified: + + + stylesheet (REQUIRED) + + + Stylesheet file. + + + + + + + + + + + + + Retrieval Facility Examples + + MARC21 backend + + A typical way to use the retrieval facility is to enable XML + for servers that only supports ISO2709 encoded MARC21 records. + + + + + + + + + + + + + + + + +]]> + + + This means that our frontend supports: + + + + MARC21 F(ull) records. + + + + + MARC21 B(rief) records. + + + + + + MARCXML records. + + + + + + Dublin core records. + + + + + + + + API + + It should be easy to use the retrieval systems from applications. Refer + to the headers + yaz/retrieval.h and + yaz/record_conv.h. + + +