From: Adam Dickmeiss Date: Thu, 19 Jul 2001 23:29:40 +0000 (+0000) Subject: Emacs indent of XML-documentation. Only minor changes in contents. X-Git-Tag: YAZ.1.8~65 X-Git-Url: http://git.indexdata.com/?p=yaz-moved-to-github.git;a=commitdiff_plain;h=ce853cc4919ab346fd629e7727905d3ee6e1129f Emacs indent of XML-documentation. Only minor changes in contents. --- diff --git a/doc/asn.xml b/doc/asn.xml index ed7dda4..e126267 100644 --- a/doc/asn.xml +++ b/doc/asn.xml @@ -1,139 +1,139 @@ - -The ASN Module -Introduction - -The &asn; module provides you with a set of C struct definitions for the -various PDUs of the protocol, as well as for the complex types -appearing within the PDUs. For the primitive data types, the C -representation often takes the form of an ordinary C language type, -such as int. For ASN.1 constructs that have no direct -representation in C, such as general octet strings and bit strings, -the &odr; module (see section The ODR Module) -provides auxiliary definitions. - - -Preparing PDUs - - -A structure representing a complex ASN.1 type doesn't in itself contain the -members of that type. Instead, the structure contains -pointers to the members of the type. -This is necessary, in part, to allow a mechanism for specifying which -of the optional structure (SEQUENCE) members are present, and which -are not. It follows that you will need to somehow provide space for -the individual members of the structure, and set the pointers to -refer to the members. - - -The conversion routines don't care how you allocate and maintain your -C structures - they just follow the pointers that you provide. -Depending on the complexity of your application, and your personal -taste, there are at least three different approaches that you may take -when you allocate the structures. - - - -You can use static or automatic local variables in the function that -prepares the PDU. This is a simple approach, and it provides the most -efficient form of memory management. While it works well for flat -PDUs like the InitReqest, it will generally not be sufficient for say, -the generation of an arbitrarily complex RPN query structure. - - -You can individually create the structure and its members using the -malloc(2) function. If you want to ensure that -the data is freed when it is no longer needed, you will have to -define a function that individually releases each member of a -structure before freeing the structure itself. - - -You can use the odr_malloc() function (see section -Using ODR for details). When you use -odr_malloc(), you can release all of the -allocated data in a single operation, independent of any pointers and -relations between the data. odr_malloc() is based on a -"nibble-memory" -scheme, in which large portions of memory are allocated, and then -gradually handed out with each call to odr_malloc(). -The next time you call odr_reset(), all of the -memory allocated since the last call is recycled for future use (actually, -it is placed on a free-list). - - -You can combine all of the methods described here. This will often be -the most practical approach. For instance, you might use -odr_malloc() to allocate an entire structure and -some of its elements, while you leave other elements pointing to global -or per-session default variables. - - - -The &asn; module provides an important aid in creating new PDUs. For -each of the PDU types (say, Z_InitRequest), a -function is provided that allocates and initializes an instance of -that PDU type for you. In the case of the InitRequest, the function is -simply named zget_InitRequest(), and it sets up -reasonable default value for all of the mandatory members. The optional -members are generally initialized to null pointers. This last aspect -is very important: it ensures that if the PDU definitions are -extended after you finish your implementation (to accommodate -new versions of the protocol, say), you won't get into trouble with -uninitialized pointers in your structures. The functions use -odr_malloc() to -allocate the PDUs and its members, so you can free everything again with a -single call to odr_reset(). We strongly recommend -that you use the zget_* -functions whenever you are preparing a PDU (in a C++ API, the -zget_ -functions would probably be promoted to constructors for the -individual types). - - -The prototype for the individual PDU types generally look like this: - - - Z_<type> *zget_<type>(ODR o); - - - -eg.: - - - - Z_InitRequest *zget_InitRequest(ODR o); - - - -The &odr; handle should generally be your encoding stream, but it needn't be. - - -As well as the individual PDU functions, a function -zget_APDU() is -provided, which allocates a toplevel Z-APDU of the type requested: - - - - Z_APDU *zget_APDU(ODR o, int which); - - - -The which parameter is (of course) the discriminator -belonging to the Z_APDU CHOICE type. -All of the interface described here is provided by the &asn; module, and -you access it through the proto.h header file. - - - -Object Identifiers - -When you refer to object identifiers in your application, you need to -be aware that SR and Z39.50 use two different set of OIDs to refer to -the same objects. To handle this easily, &yaz; provides a utility module -to &asn; which provides an internal representation of the OIDs used in -both protocols. Each oid is described by a structure: - - - + + The ASN Module + Introduction + + The &asn; module provides you with a set of C struct definitions for the + various PDUs of the protocol, as well as for the complex types + appearing within the PDUs. For the primitive data types, the C + representation often takes the form of an ordinary C language type, + such as int. For ASN.1 constructs that have no direct + representation in C, such as general octet strings and bit strings, + the &odr; module (see section The ODR Module) + provides auxiliary definitions. + + + Preparing PDUs + + + A structure representing a complex ASN.1 type doesn't in itself contain the + members of that type. Instead, the structure contains + pointers to the members of the type. + This is necessary, in part, to allow a mechanism for specifying which + of the optional structure (SEQUENCE) members are present, and which + are not. It follows that you will need to somehow provide space for + the individual members of the structure, and set the pointers to + refer to the members. + + + The conversion routines don't care how you allocate and maintain your + C structures - they just follow the pointers that you provide. + Depending on the complexity of your application, and your personal + taste, there are at least three different approaches that you may take + when you allocate the structures. + + + + You can use static or automatic local variables in the function that + prepares the PDU. This is a simple approach, and it provides the most + efficient form of memory management. While it works well for flat + PDUs like the InitReqest, it will generally not be sufficient for say, + the generation of an arbitrarily complex RPN query structure. + + + You can individually create the structure and its members using the + malloc(2) function. If you want to ensure that + the data is freed when it is no longer needed, you will have to + define a function that individually releases each member of a + structure before freeing the structure itself. + + + You can use the odr_malloc() function (see section + Using ODR for details). When you use + odr_malloc(), you can release all of the + allocated data in a single operation, independent of any pointers and + relations between the data. odr_malloc() is based on a + "nibble-memory" + scheme, in which large portions of memory are allocated, and then + gradually handed out with each call to odr_malloc(). + The next time you call odr_reset(), all of the + memory allocated since the last call is recycled for future use (actually, + it is placed on a free-list). + + + You can combine all of the methods described here. This will often be + the most practical approach. For instance, you might use + odr_malloc() to allocate an entire structure and + some of its elements, while you leave other elements pointing to global + or per-session default variables. + + + + The &asn; module provides an important aid in creating new PDUs. For + each of the PDU types (say, Z_InitRequest), a + function is provided that allocates and initializes an instance of + that PDU type for you. In the case of the InitRequest, the function is + simply named zget_InitRequest(), and it sets up + reasonable default value for all of the mandatory members. The optional + members are generally initialized to null pointers. This last aspect + is very important: it ensures that if the PDU definitions are + extended after you finish your implementation (to accommodate + new versions of the protocol, say), you won't get into trouble with + uninitialized pointers in your structures. The functions use + odr_malloc() to + allocate the PDUs and its members, so you can free everything again with a + single call to odr_reset(). We strongly recommend + that you use the zget_* + functions whenever you are preparing a PDU (in a C++ API, the + zget_ + functions would probably be promoted to constructors for the + individual types). + + + The prototype for the individual PDU types generally look like this: + + + Z_<type> *zget_<type>(ODR o); + + + + eg.: + + + + Z_InitRequest *zget_InitRequest(ODR o); + + + + The &odr; handle should generally be your encoding stream, but it needn't be. + + + As well as the individual PDU functions, a function + zget_APDU() is + provided, which allocates a toplevel Z-APDU of the type requested: + + + + Z_APDU *zget_APDU(ODR o, int which); + + + + The which parameter is (of course) the discriminator + belonging to the Z_APDU CHOICE type. + All of the interface described here is provided by the &asn; module, and + you access it through the proto.h header file. + + + + Object Identifiers + + When you refer to object identifiers in your application, you need to + be aware that SR and Z39.50 use two different set of OIDs to refer to + the same objects. To handle this easily, &yaz; provides a utility module + to &asn; which provides an internal representation of the OIDs used in + both protocols. Each oid is described by a structure: + + + typedef struct oident { enum oid_proto proto; @@ -142,63 +142,63 @@ typedef struct oident int oidsuffix[OID_SIZE]; char *desc; } oident; - + - -The proto field can be set to either -PROTO_SR or PROTO_Z3950. -The class might be, say, -CLASS_RECSYN, and the value might be -VAL_USMARC for the USMARC record format. Functions - + + The proto field can be set to either + PROTO_SR or PROTO_Z3950. + The class might be, say, + CLASS_RECSYN, and the value might be + VAL_USMARC for the USMARC record format. Functions + - + int *oid_ent_to_oid(struct oident *ent, int *dst); struct oident *oid_getentbyoid(int *o); - - - -are provided to map between object identifiers and database entries. -If you store a member of the oid_proto type in -your association state information, it's a simple matter, at runtime, -to generate the correct OID when you need it. For decoding, you can -simply ignore the proto field, or if you're strict, you can verify -that your peer is using the OID family from the correct protocol. -The desc field is a short, human-readable name -for the PDU, useful mainly for diagnostic output. - - - - -The old function oid_getoidbyent still exists but is -not thread safe. Use oid_ent_to_oid instead -and pass an array of size OID_SIZE. - - - - - -Plans are underway to merge the two protocols into a single -definition, with one set of object identifiers. When this happens, the -oid module will no longer be required to support protocol -independence, but it should still be useful as a simple OID database. - - - - -EXTERNAL Data - - -In order to achieve extensibility and adaptability to different -application domains, the new version of the protocol defines many -structures outside of the main ASN.1 specification, referencing them -through ASN.1 EXTERNAL constructs. To simplify the construction and access -to the externally referenced data, the &asn; module defines a -specialized version of the EXTERNAL construct, called -Z_External.It is defined thus: - - - + + + + are provided to map between object identifiers and database entries. + If you store a member of the oid_proto type in + your association state information, it's a simple matter, at runtime, + to generate the correct OID when you need it. For decoding, you can + simply ignore the proto field, or if you're strict, you can verify + that your peer is using the OID family from the correct protocol. + The desc field is a short, human-readable name + for the PDU, useful mainly for diagnostic output. + + + + + The old function oid_getoidbyent still exists but is + not thread safe. Use oid_ent_to_oid instead + and pass an array of size OID_SIZE. + + + + + + Plans are underway to merge the two protocols into a single + definition, with one set of object identifiers. When this happens, the + oid module will no longer be required to support protocol + independence, but it should still be useful as a simple OID database. + + + + + EXTERNAL Data + + + In order to achieve extensibility and adaptability to different + application domains, the new version of the protocol defines many + structures outside of the main ASN.1 specification, referencing them + through ASN.1 EXTERNAL constructs. To simplify the construction and access + to the externally referenced data, the &asn; module defines a + specialized version of the EXTERNAL construct, called + Z_External.It is defined thus: + + + typedef struct Z_External { Odr_oid *direct_reference; @@ -206,527 +206,544 @@ typedef struct Z_External char *descriptor; enum { - /* Generic types */ - Z_External_single = 0, - Z_External_octet, - Z_External_arbitrary, + /* Generic types */ + Z_External_single = 0, + Z_External_octet, + Z_External_arbitrary, - /* Specific types */ - Z_External_SUTRS, - Z_External_explainRecord, - Z_External_resourceReport1, - Z_External_resourceReport2 + /* Specific types */ + Z_External_SUTRS, + Z_External_explainRecord, + Z_External_resourceReport1, + Z_External_resourceReport2 - ... + ... } which; union { - /* Generic types */ - Odr_any *single_ASN1_type; - Odr_oct *octet_aligned; - Odr_bitmask *arbitrary; + /* Generic types */ + Odr_any *single_ASN1_type; + Odr_oct *octet_aligned; + Odr_bitmask *arbitrary; - /* Specific types */ - Z_SUTRS *sutrs; - Z_ExplainRecord *explainRecord; - Z_ResourceReport1 *resourceReport1; - Z_ResourceReport2 *resourceReport2; + /* Specific types */ + Z_SUTRS *sutrs; + Z_ExplainRecord *explainRecord; + Z_ResourceReport1 *resourceReport1; + Z_ResourceReport2 *resourceReport2; - ... + ... } u; } Z_External; - - - -When decoding, the &asn; module will attempt to determine which -syntax describes the data by looking at the reference fields -(currently only the direct-reference). For ASN.1 structured data, you -need only consult the which field to determine the type of -data. You can the access the data directly through the union. When -constructing data for encoding, you set the union pointer to point to -the data, and set the which field accordingly. -Remember also to set the direct (or indirect) reference to the correct -OID for the data type. -For non-ASN.1 data such as MARC records, use the -octet_aligned arm of the union. - - - -Some servers return ASN.1 structured data values (eg. database -records) as BER-encoded records placed in the octet-aligned -branch of the EXTERNAL CHOICE. The ASN-module will not -automatically decode these records. To help you decode the records in -the application, the function - - - -Z_ext_typeent *z_ext_gettypebyref(oid_value ref); - - - -Can be used to retrieve information about the known, external data -types. The function return a pointer to a static area, or NULL, if no -match for the given direct reference is found. The -Z_ext_typeent -is defined as: - - - + + + + When decoding, the &asn; module will attempt to determine which + syntax describes the data by looking at the reference fields + (currently only the direct-reference). For ASN.1 structured data, you + need only consult the which field to determine the + type of data. You can the access the data directly through the union. + When constructing data for encoding, you set the union pointer to point + to the data, and set the which field accordingly. + Remember also to set the direct (or indirect) reference to the correct + OID for the data type. + For non-ASN.1 data such as MARC records, use the + octet_aligned arm of the union. + + + + Some servers return ASN.1 structured data values (eg. database + records) as BER-encoded records placed in the + octet-aligned branch of the EXTERNAL CHOICE. + The ASN-module will not automatically decode + these records. To help you decode the records in the application, the + function + + + + Z_ext_typeent *z_ext_gettypebyref(oid_value ref); + + + + Can be used to retrieve information about the known, external data + types. The function return a pointer to a static area, or NULL, if no + match for the given direct reference is found. The + Z_ext_typeent + is defined as: + + + typedef struct Z_ext_typeent { oid_value dref; /* the direct-reference OID value. */ int what; /* discriminator value for the external CHOICE */ Odr_fun fun; /* decoder function */ } Z_ext_typeent; - - - -The what member contains the Z_External -union discriminator value for the given type: For the SUTRS record -syntax, the value would be Z_External_sutrs. -The fun member contains a pointer to the -function which encodes/decodes the given type. Again, for the SUTRS -record syntax, the value of fun would be -z_SUTRS (a function pointer). - - - -If you receive an EXTERNAL which contains an octet-string value that -you suspect of being an ASN.1-structured data value, you can use -z_ext_gettypebyref to look for the provided -direct-reference. -If the return value is different from NULL, you can use the provided -function to decode the BER string (see section -Using ODR). - - - -If you want to send EXTERNALs containing -ASN.1-structured values in the occtet-aligned branch of the CHOICE, this -is possible too. However, on the encoding phase, it requires a somewhat -involved juggling around of the various buffers involved. - - -If you need to add new, externally defined data types, you must update -the struct above, in the source file prt-ext.h, as -well as the encoder/decoder in the file prt-ext.c. -When changing the latter, remember to update both the arm -arrary and the list type_table, which drives the CHOICE -biasing that is necessary to tell the different, structured types apart -on decoding. - - - - -Eventually, the EXTERNAL processing will most likely -automatically insert the correct OIDs or indirect-refs. First, -however, we need to determine how application-context management -(specifically the presentation-context-list) should fit into the -various modules. - - - - -PDU Contents Table - - -We include, for reference, a listing of the fields of each top-level -PDU, as well as their default settings. - - -Default settings for PDU Initialize Request - - - - - - -Field -Type -Default Value - - - - - -referenceIdZ_ReferenceIdNULL - - - -protocolVersionOdr_bitmaskEmpty bitmask - - - -optionsOdr_bitmaskEmpty bitmask - - - -preferredMessageSizeint30*1024 - - - -maximumRecordSizeint30*1024 - - - -idAuthenticationZ_IdAuthenticationNULL - - - -implementationIdchar*"YAZ (id=81)" - - - -implementationNamechar*"Index Data/YAZ" - - - -implementationVersionchar*YAZ_VERSION - - - -userInformationFieldZ_UserInformationNULL - - - -otherInfoZ_OtherInformationNULL - - - - -
- -Default settings for PDU Initialize Response - - - - - - -Field -Type -Default Value - - - - - -referenceIdZ_ReferenceIdNULL - - - -protocolVersionOdr_bitmaskEmpty bitmask - - - -optionsOdr_bitmaskEmpty bitmask - - - -preferredMessageSizeint30*1024 - - - -maximumRecordSizeint30*1024 - - - -resultbool_tTRUE - - - -implementationIdchar*"YAZ (id=81)" - - - -implementationNamechar*"Index Data/YAZ" - - - -implementationVersionchar*YAZ_VERSION - - - -userInformationFieldZ_UserInformationNULL - - - -otherInfoZ_OtherInformationNULL - - - - -
- -Default settings for PDU Search Request - - - - - - -Field -Type -Default Value - - - - - -referenceIdZ_ReferenceIdNULL - - - -smallSetUpperBoundint0 - - - -largeSetLowerBoundint1 - - - -mediumSetPresentNumberint0 - - - -replaceIndicatorbool_tTRUE - - - -resultSetNamechar *"default" - - - -num_databaseNamesint0 - - - -databaseNameschar **NULL - - - -smallSetElementSetNamesZ_ElementSetNamesNULL - - - -mediumSetElementSetNamesZ_ElementSetNamesNULL - - - -preferredRecordSyntaxOdr_oidNULL - - - -queryZ_QueryNULL - - - -additionalSearchInfoZ_OtherInformationNULL - - - -otherInfoZ_OtherInformationNULL - - - - -
- - -Z_SearchResponse ----------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -resultCount int 0 -numberOfRecordsReturned int 0 -nextResultSetPosition int 0 -searchStatus bool_t TRUE -resultSetStatus int NULL -presentStatus int NULL -records Z_Records NULL -additionalSearchInfo Z_OtherInformation NULL -otherInfo Z_OtherInformation NULL - - - -Z_PresentRequest ----------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -resultSetId char* "default" -resultSetStartPoint int 1 -numberOfRecordsRequested int 10 -num_ranges int 0 -additionalRanges Z_Range NULL -recordComposition Z_RecordComposition NULL -preferredRecordSyntax Odr_oid NULL -maxSegmentCount int NULL -maxRecordSize int NULL -maxSegmentSize int NULL -otherInfo Z_OtherInformation NULL - - - -Z_PresentResponse ------------------ -Field Type Default value - -referenceId Z_ReferenceId NULL -numberOfRecordsReturned int 0 -nextResultSetPosition int 0 -presentStatus int Z_PRES_SUCCESS -records Z_Records NULL -otherInfo Z_OtherInformation NULL - - - -Z_DeleteResultSetRequest ------------------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -deleteFunction int Z_DeleteRequest_list -num_ids int 0 -resultSetList char** NULL -otherInfo Z_OtherInformation NULL - - - -Z_DeleteResultSetResponse -------------------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -deleteOperationStatus int Z_DeleteStatus_success -num_statuses int 0 -deleteListStatuses Z_ListStatus** NULL -numberNotDeleted int NULL -num_bulkStatuses int 0 -bulkStatuses Z_ListStatus NULL -deleteMessage char* NULL -otherInfo Z_OtherInformation NULL - - - -Z_ScanRequest -------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -num_databaseNames int 0 -databaseNames char** NULL -attributeSet Odr_oid NULL -termListAndStartPoint Z_AttributesPlus... NULL -stepSize int NULL -numberOfTermsRequested int 20 -preferredPositionInResponse int NULL -otherInfo Z_OtherInformation NULL - - - -Z_ScanResponse --------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -stepSize int NULL -scanStatus int Z_Scan_success -numberOfEntriesReturned int 0 -positionOfTerm int NULL -entries Z_ListEntris NULL -attributeSet Odr_oid NULL -otherInfo Z_OtherInformation NULL - - - -Z_TriggerResourceControlRequest -------------------------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -requestedAction int Z_TriggerResourceCtrl_resou.. -prefResourceReportFormat Odr_oid NULL -resultSetWanted bool_t NULL -otherInfo Z_OtherInformation NULL - - - -Z_ResourceControlRequest ------------------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -suspendedFlag bool_t NULL -resourceReport Z_External NULL -partialResultsAvailable int NULL -responseRequired bool_t FALSE -triggeredRequestFlag bool_t NULL -otherInfo Z_OtherInformation NULL - - - -Z_ResourceControlResponse -------------------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -continueFlag bool_t TRUE -resultSetWanted bool_t NULL -otherInfo Z_OtherInformation NULL - - - -Z_AccessControlRequest ----------------------- -Field Type Default value - -referenceId Z_ReferenceId NULL -which enum Z_AccessRequest_simpleForm; -u union NULL -otherInfo Z_OtherInformation NULL - - - -Z_AccessControlResponse ------------------------ -Field Type Default value - -referenceId Z_ReferenceId NULL -which enum Z_AccessResponse_simpleForm -u union NULL -diagnostic Z_DiagRec NULL -otherInfo Z_OtherInformation NULL - - - -Z_Segment ---------- -Field Type Default value - -referenceId Z_ReferenceId NULL -numberOfRecordsReturned int value=0 -num_segmentRecords int 0 -segmentRecords Z_NamePlusRecord NULL -otherInfo Z_OtherInformation NULL - - - -Z_Close -------- -Field Type Default value - -referenceId Z_ReferenceId NULL -closeReason int Z_Close_finished -diagnosticInformation char* NULL -resourceReportFormat Odr_oid NULL -resourceFormat Z_External NULL -otherInfo Z_OtherInformation NULL - - - - - + + + + The what member contains the + Z_External union discriminator value for the + given type: For the SUTRS record syntax, the value would be + Z_External_sutrs. + The fun member contains a pointer to the + function which encodes/decodes the given type. Again, for the SUTRS + record syntax, the value of fun would be + z_SUTRS (a function pointer). + + + + If you receive an EXTERNAL which contains an octet-string value that + you suspect of being an ASN.1-structured data value, you can use + z_ext_gettypebyref to look for the provided + direct-reference. + If the return value is different from NULL, you can use the provided + function to decode the BER string (see section + Using ODR). + + + + If you want to send EXTERNALs containing + ASN.1-structured values in the occtet-aligned branch of the CHOICE, this + is possible too. However, on the encoding phase, it requires a somewhat + involved juggling around of the various buffers involved. + + + If you need to add new, externally defined data types, you must update + the struct above, in the source file prt-ext.h, as + well as the encoder/decoder in the file prt-ext.c. + When changing the latter, remember to update both the arm + arrary and the list type_table, which drives the CHOICE + biasing that is necessary to tell the different, structured types apart + on decoding. + + + + + Eventually, the EXTERNAL processing will most likely + automatically insert the correct OIDs or indirect-refs. First, + however, we need to determine how application-context management + (specifically the presentation-context-list) should fit into the + various modules. + + + + + PDU Contents Table + + + We include, for reference, a listing of the fields of each top-level + PDU, as well as their default settings. + + + Default settings for PDU Initialize Request + + + + + + + Field + Type + Default Value + + + + + + referenceIdZ_ReferenceIdNULL + + + + protocolVersionOdr_bitmaskEmpty bitmask + + + + optionsOdr_bitmaskEmpty bitmask + + + + preferredMessageSizeint30*1024 + + + + maximumRecordSizeint30*1024 + + + + idAuthenticationZ_IdAuthenticationNULL + + + + implementationIdchar*"YAZ (id=81)" + + + + implementationNamechar*"Index Data/YAZ" + + + + implementationVersionchar*YAZ_VERSION + + + + userInformationFieldZ_UserInformationNULL + + + + otherInfoZ_OtherInformationNULL + + + + +
+ + Default settings for PDU Initialize Response + + + + + + + Field + Type + Default Value + + + + + + referenceIdZ_ReferenceIdNULL + + + + protocolVersionOdr_bitmaskEmpty bitmask + + + + optionsOdr_bitmaskEmpty bitmask + + + + preferredMessageSizeint30*1024 + + + + maximumRecordSizeint30*1024 + + + + resultbool_tTRUE + + + + implementationIdchar*"YAZ (id=81)" + + + + implementationNamechar*"Index Data/YAZ" + + + + implementationVersionchar*YAZ_VERSION + + + + userInformationFieldZ_UserInformationNULL + + + + otherInfoZ_OtherInformationNULL + + + + +
+ + Default settings for PDU Search Request + + + + + + + Field + Type + Default Value + + + + + + referenceIdZ_ReferenceIdNULL + + + + smallSetUpperBoundint0 + + + + largeSetLowerBoundint1 + + + + mediumSetPresentNumberint0 + + + + replaceIndicatorbool_tTRUE + + + + resultSetNamechar *"default" + + + + num_databaseNamesint0 + + + + databaseNameschar **NULL + + + + smallSetElementSetNamesZ_ElementSetNamesNULL + + + + mediumSetElementSetNamesZ_ElementSetNamesNULL + + + + preferredRecordSyntaxOdr_oidNULL + + + + queryZ_QueryNULL + + + + additionalSearchInfoZ_OtherInformationNULL + + + + otherInfoZ_OtherInformationNULL + + + + +
+ + + Z_SearchResponse + ---------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + resultCount int 0 + numberOfRecordsReturned int 0 + nextResultSetPosition int 0 + searchStatus bool_t TRUE + resultSetStatus int NULL + presentStatus int NULL + records Z_Records NULL + additionalSearchInfo Z_OtherInformation NULL + otherInfo Z_OtherInformation NULL + + + + Z_PresentRequest + ---------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + resultSetId char* "default" + resultSetStartPoint int 1 + numberOfRecordsRequested int 10 + num_ranges int 0 + additionalRanges Z_Range NULL + recordComposition Z_RecordComposition NULL + preferredRecordSyntax Odr_oid NULL + maxSegmentCount int NULL + maxRecordSize int NULL + maxSegmentSize int NULL + otherInfo Z_OtherInformation NULL + + + + Z_PresentResponse + ----------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + numberOfRecordsReturned int 0 + nextResultSetPosition int 0 + presentStatus int Z_PRES_SUCCESS + records Z_Records NULL + otherInfo Z_OtherInformation NULL + + + + Z_DeleteResultSetRequest + ------------------------ + Field Type Default value + + referenceId Z_ReferenceId NULL + deleteFunction int Z_DeleteRequest_list + num_ids int 0 + resultSetList char** NULL + otherInfo Z_OtherInformation NULL + + + + Z_DeleteResultSetResponse + ------------------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + deleteOperationStatus int Z_DeleteStatus_success + num_statuses int 0 + deleteListStatuses Z_ListStatus** NULL + numberNotDeleted int NULL + num_bulkStatuses int 0 + bulkStatuses Z_ListStatus NULL + deleteMessage char* NULL + otherInfo Z_OtherInformation NULL + + + + Z_ScanRequest + ------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + num_databaseNames int 0 + databaseNames char** NULL + attributeSet Odr_oid NULL + termListAndStartPoint Z_AttributesPlus... NULL + stepSize int NULL + numberOfTermsRequested int 20 + preferredPositionInResponse int NULL + otherInfo Z_OtherInformation NULL + + + + Z_ScanResponse + -------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + stepSize int NULL + scanStatus int Z_Scan_success + numberOfEntriesReturned int 0 + positionOfTerm int NULL + entries Z_ListEntris NULL + attributeSet Odr_oid NULL + otherInfo Z_OtherInformation NULL + + + + Z_TriggerResourceControlRequest + ------------------------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + requestedAction int Z_TriggerResourceCtrl_resou.. + prefResourceReportFormat Odr_oid NULL + resultSetWanted bool_t NULL + otherInfo Z_OtherInformation NULL + + + + Z_ResourceControlRequest + ------------------------ + Field Type Default value + + referenceId Z_ReferenceId NULL + suspendedFlag bool_t NULL + resourceReport Z_External NULL + partialResultsAvailable int NULL + responseRequired bool_t FALSE + triggeredRequestFlag bool_t NULL + otherInfo Z_OtherInformation NULL + + + + Z_ResourceControlResponse + ------------------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + continueFlag bool_t TRUE + resultSetWanted bool_t NULL + otherInfo Z_OtherInformation NULL + + + + Z_AccessControlRequest + ---------------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + which enum Z_AccessRequest_simpleForm; + u union NULL + otherInfo Z_OtherInformation NULL + + + + Z_AccessControlResponse + ----------------------- + Field Type Default value + + referenceId Z_ReferenceId NULL + which enum Z_AccessResponse_simpleForm + u union NULL + diagnostic Z_DiagRec NULL + otherInfo Z_OtherInformation NULL + + + + Z_Segment + --------- + Field Type Default value + + referenceId Z_ReferenceId NULL + numberOfRecordsReturned int value=0 + num_segmentRecords int 0 + segmentRecords Z_NamePlusRecord NULL + otherInfo Z_OtherInformation NULL + + + + Z_Close + ------- + Field Type Default value + + referenceId Z_ReferenceId NULL + closeReason int Z_Close_finished + diagnosticInformation char* NULL + resourceReportFormat Odr_oid NULL + resourceFormat Z_External NULL + otherInfo Z_OtherInformation NULL + + + + + + + diff --git a/doc/comstack.xml b/doc/comstack.xml index 7ecabfc..86b88f6 100644 --- a/doc/comstack.xml +++ b/doc/comstack.xml @@ -1,9 +1,9 @@ - -The COMSTACK Module + + The COMSTACK Module -Synopsis (blocking mode) + Synopsis (blocking mode) - + COMSTACK *stack; char *buf = 0; @@ -48,692 +48,691 @@ if (!length_incoming) { cs_close(stack); if (buf) free(buf); + + + + + Introduction + + + The &comstack; + subsystem provides a transparent interface to different types of transport + stacks for the exchange of BER-encoded data. At present, the + RFC1729 method (BER over TCP/IP), and Peter Furniss' XTImOSI + stack are supported, but others may be added in time. The philosophy of the + module is to provide a simple interface by hiding unused options and + facilities of the underlying libraries. This is always done at the risk + of losing generality, and it may prove that the interface will need + extension later on. + + + + The interface is implemented in such a fashion that only the + sub-layers constructed to the transport methods that you wish to + use in your application are linked in. + + + + You will note that even though simplicity was a goal in the design, + the interface is still orders of magnitudes more complex than the + transport systems found in many other packages. One reason is that + the interface needs to support the somewhat different requirements of + the different lower-layer communications stacks; another important reason is + that the interface seeks to provide a more or less industrial-strength + approach to asynchronous event-handling. When no function is allowed + to block, things get more complex - particularly on the server + side. We urge you to have a look at the demonstration client and server + provided with the package. They are meant to be easily readable and + instructive, while still being at least moderately useful. + + + + Common Functions + + Managing Endpoints + + + COMSTACK cs_create(CS_TYPE type, int blocking, int protocol); + + + + Creates an instance of the protocol stack - a communications endpoint. + The type parameter determines the mode of communication. + At present, the values + tcpip_type + and + mosi_type + are recognized. The function returns a null-pointer if a system error + occurs. The blocking parameter should be one if you wish + the association to operate in blocking mode, zero otherwise. The + protocol field should be one of + PROTO_SR or PROTO_Z3950. + + + + + int cs_close(COMSTACK handle); + + + + Closes the connection (as elegantly as the lower layers will permit), + and releases the resouces pointed to by the + handle + parameter. The + handle + should not be referenced again after this call. + + + + + We really need a soft disconnect, don't we? + + + + + Data Exchange + + + int cs_put(COMSTACK handle, char *buf, int len); + + + + Sends + buf + down the wire. In blocking mode, this function will return only when a + full buffer has been written, or an error has occurred. In nonblocking + mode, it's possible that the function will be unable to send the full + buffer at once, which will be indicated by a return value of 1. The + function will keep track of the number of octets already written; you + should call it repeatedly with the same values of buf + and len, until the buffer has been transmitted. + When a full buffer has been sent, the function will return 0 for + success. -1 indicates an error condition (see below). + + + + int cs_get(COMSTACK handle, char **buf, int *size); + + + + Receives a PDU from the peer. Returns the number of bytes + read. In nonblocking mode, it is possible that not all of the packet can be + read at once. In this case, the function returns 1. To simplify the + interface, the function is + responsible for managing the size of the buffer. It will be reallocated + if necessary to contain large packages, and will sometimes be moved + around internally by the subsystem when partial packages are read. Before + calling + cs_get + for the fist time, the buffer can be initialized to the null pointer, + and the length should also be set to 0 - cs_get will perform a + malloc(2) + on the buffer for you. When a full buffer has been read, the size of + the package is returned (which will always be greater than 1). -1 + indicates an error condition. + + + + See also the cs_more() function below. + + + + int cs_more(COMSTACK handle); + + + + The cs_more() function should be used in conjunction + with cs_get and + select(2). + The cs_get() function will sometimes + (notably in the TCP/IP mode) read more than a single protocol package + off the network. When this happens, the extra package is stored + by the subsystem. After callig cs_get(), and before + waiting for more input, You should always call + cs_more() + to check if there's a full protocol package already read. If + cs_more() + returns 1, + cs_get() + can be used to immediately fetch the new package. For the + mOSI + subsystem, the function should always return 0, but if you want your + stuff to be protocol independent, you should use it. + + + + + The cs_more() + function is required because the RFC1729-method + does not provide a way of separating individual PDUs, short of + partially decoding the BER. Some other implementations will carefully + nibble at the packet by calling + read(2) + several times. This was felt to be too inefficient (or at least + clumsy) - hence the call for this extra function. + + + + + int cs_look(COMSTACK handle); + + + + This function is useful when you're operating in nonblocking + mode. Call it when + select(2) + tells you there's something happening on the line. It returns one of + the following values: + + + + CS_NONE + No event is pending. The data found on the line was not a complete package. + + + CS_CONNECT + A response to your connect request has been received. Call + cs_rcvconnect + to process the event and to finalize the connection establishment. + + + CS_DISCON + The other side has closed the connection (or maybe sent a disconnect + request - but do we care? Maybe later). Call + cs_close to close your end of the association as well. + + + CS_LISTEN + A connect request has been received. Call cs_listen + to process the event. + + + CS_DATA + There's data to be found on the line. Call cs_get + to get it. + + + + + + You should be aware that even if + cs_look() + tells you that there's an event event pending, the corresponding + function may still return and tell you there was nothing to be found. + This means that only part of a package was available for reading. The + same event will show up again, when more data has arrived. + + + + + int cs_fileno(COMSTACK h); + + + + Returns the file descriptor of the association. Use this when + file-level operations on the endpoint are required + (select(2) operations, specifically). + + + + + + Client Side + + + int cs_connect(COMSTACK handle, void *address); + + + + Initiate a connection with the target at address + (more onaddresses below). The function will return 0 on success, and 1 if + the operation does not complete immediately (this will only + happen on a nonblocking endpoint). In this case, use + cs_rcvconnect to complete the operation, + when select(2) reports input pending on the + association. + + + + int cs_rcvconnect(COMSTACK handle); + + + + Complete a connect operation initiated by cs_connect(). + It will return 0 on success; 1 if the operation has not yet completed (in + this case, call the function again later); -1 if an error has occured. + + + + + Server Side + + + To establish a server under the inetd server, you + can use + + + + COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking, + int protocol); + + + + The socket parameter is an established socket (when + your application is invoked from inetd, the + socket will typically be 0. + The following parameters are identical to the ones for + cs_create. + + + + int cs_bind(COMSTACK handle, void *address, int mode) + + + + Binds a local address to the endpoint. Read about addresses below. The + mode parameter should be either + CS_CLIENT or CS_SERVER. + + + + int cs_listen(COMSTACK handle, char *addr, int *addrlen); + + + + Call this to process incoming events on an endpoint that has been + bound in listening mode. It will return 0 to indicate that the connect + request has been received, 1 to signal a partial reception, and -1 to + indicate an error condition. + + + + COMSTACK cs_accept(COMSTACK handle); + + + + This finalises the server-side association establishment, after + cs_listen has completed successfully. It returns a new connection + endpoint, which represents the new association. The application will + typically wish to fork off a process to handle the association at this + point, and continue listen for new connections on the old + handle. + + + + You can use the call + + + + char *cs_addrstr(COMSTACK); + + + + on an established connection to retrieve the hostname of the remote host. + + + + You may need to use this function with some care if your + name server service is slow or unreliable + + + + + Addresses + + + The low-level format of the addresses are different depending on the + mode of communication you have chosen. A function is provided by each + of the lower layers to map a user-friendly string-form address to the + binary form required by the lower layers. + + + + struct sockaddr_in *tcpip_strtoaddr(char *str); + + struct netbuf *mosi_strtoaddr(char *str); + + + + The format for TCP/IP addresses is straightforward: + + + + <host> [ ':' <portnum> ] + + + + The hostname can be either a domain name or an IP address. + The port number, if omitted, defaults to 210. + + + + For OSI, the format is + + + + [ <t-selector> '/' ] <host> [ ':' <port> ] + + + + The transport selector is given as an even number of hex digits. + + + + You'll note that the address format for the OSI mode are just a subset + of full presentation addresses. We use presentation addresses because + xtimosi doesn't, in itself, allow access to the X.500 Directory + service. We use a limited form, because we haven't yet come across an + implementation that used more of the elements of a full p-address. It + is a fairly simple matter to add the rest of the elements to the + address format as needed, however: Xtimosi does + support the full P-address structure. + + + + In both transport modes, the special hostname "@" is mapped + to any local address (the manifest constant INADDR_ANY). + It is used to establish local listening endpoints in the server role. + + + + When a connection has been established, you can use + + + + char cs_addrstr(COMSTACK h); + + + + to retrieve the host name of the peer system. The function returns a pointer + to a static area, which is overwritten on the next call to the function. + + + + + We have left the issue of X.500 name-to-address mapping open, for the + moment. It would be a simple matter to provide a table-based mapping, + if desired. Alternately, we could use the X.500 client-function that + is provided with the ISODE (although this would defeat some of the + purpose of using ThinOSI in the first place. We have been told that it + should be within the realm of the possible to implement a lightweight + implementation of the necessary X.500 client capabilities on top of + ThinOSI. This would be the ideal solution, we feel. On the other hand, it + still remains to be seen just what role the Directory will play in a world + populated by ThinOSI and other pragmatic solutions. + + + + + + Diagnostics + + + All functions return -1 if an error occurs. Typically, the functions + will return 0 on success, but the data exchange functions + (cs_get, cs_put, + cs_more) follow special rules. Consult their + descriptions. + + + + When a function (including the data exchange functions) reports an + error condition, use the function + cs_errno() to determine the cause of the + problem. The function + + + + void cs_perror(COMSTACK handle char *message); + + + + works like perror(2) and prints the + message argument, along with a system message, to + stderr. Use the character array + + + + extern const char *cs_errlist[]; + + + + to get hold of the message, if you want to process it differently. + The function + + + + const char *cs_stackerr(COMSTACK handle); + + + + Returns an error message from the lower layer, if one has been + provided. + + + + Enabling OSI Communication + + Installing Xtimosi + + Although you will have to download Peter Furniss' XTI/mOSI + implementation for yourself, we've tried to make the integration as + simple as possible. + + + + The latest version of xtimosi will generally be under + + + + ftp://pluto.ulcc.ac.uk/ulcc/thinosi/xtimosi/ + + + + When you have downloaded and unpacked the archive, it will (we assume) + have created a directory called xtimosi. + We suggest that you place this directory in the same + directory where you unpacked the &yaz; + distribution. This way, you shouldn't have to fiddle with the + makefiles of &yaz; beyond uncommenting a few lines. + + + + Go to xtimosi/src, and type "make libmosi.a/". + This should generally create the library, ready to use. + + + + + The currently available release of xtimosi has some inherent + problems that make it disfunction on certain platforms - eg. the + Digital OSF/1 workstations. It is supposedly primarily a + compiler problem, and we hope to see a release that is generally + portable. While we can't guarantee that it can be brought to work + on your platform, we'll be happy to talk to you about problems + that you might see, and relay information to the author of the + software. There are some signs that the gcc + compiler is more likely to produce a fully functional library, but this + hasn't been verified (we think that the problem is limited to the use + of hexadecimal escape-codes used in strings, which are silently + ignored by some compilers). + + + A problem has been encountered in the communication with + ISODE-based applications. If the ISODE presentation-user calls + PReadRequest() with a timeout value different + from OK or NOTOK, + he will get an immediate TIMEOUT abort when receiving large (>2041 + bytes, which is the SPDU-size that the ISODE likes to work with) packages + from an xtimosi-based implementation (probably most + other implementations as well, in fact). It seems to be a flaw in the + ISODE API, and the workaround (for ISODE users) is to either not + use an explicit timeout (switching to either blocking or + nonblocking mode), or to check that the timer really has expired + before closing the connection. + + + + + The next step in the installation is to modify the makefile in the toplevel + &yaz; + directory. The place to change is in the top of the file, and is + clearly marked with a comment. + + + + Now run make in the &yaz; toplevel directory (do a + make clean first, if the system has been previously + made without OSI support). Use the &yaz; + yaz-ztest and yaz-client + demo programs to verify that OSI communication works OK. Then, you can go + ahead and try to talk to other implementations. + + + + + Our interoperability experience is limited to version + 7 of the Nordic SR-Nett package, which has had several + protocol errors fixed from the earlier releases. If you have + problems or successes in interoperating with other + implementations, we'd be glad to hear about it, or to help + you make things work, as our resources allow. + + + + + If you write your own applications based on &yaz;, and you wish to + include OSI support, the procedure is equally simple. You should + include the xmosi.h header file in addition to + comstack.h. xmosi.h + will define the manifest constant mosi_type, which you + should pass to the cs_create() function. In + addition, you should use the function mosi_strtoaddr() + rather than tcpip_strtoaddr() when you need to + prepare an address. + + + + When you link your application, you should include (after the + libyaz.a library) the libmosi.a + library, and the librfc.a library provided with + &yaz; (for OSI transport). + + + As always, it can be very useful, if not essential, to have a look at the + example applications to see how things are done. + + + + OSI Transport + + + Xtimosi requires an implementation of the OSI transport service under + the X/OPEN XTI API. We provide an implementation of the RFC1006 + encapsulation of OSI/TP0 in TCP/IP (through the Berkeley Sockets API), + as an independent part of &yaz; (it's found under the + rfc1006 directory). + If you have access to an OSI transport provider under XTI, + you should be able to make that work too, although it may require + tinkering with the mosi_strtoaddr() function. + + + + Presentation Context Management + + + To simplify the implementation, we use Peter Furniss' alternative (PRF) + option format + for the Control of the presentation negotiation phase. This format + is enabled by default when you + compile xtimosi. + + + + The current version of &yaz; does not support + presentation-layer negotiation of response record formats. The primary + reason is that we have had access to no other SR or Z39.50 + implementations over OSI that used this + method. Secondarily, we believe that the EXPLAIN facility is a superior + mechanism for relaying target capabilities in this respect. This is not to + say that we have no intentions of supporting presentation context + negotiation - we have just hitherto given it a lower priority than other + aspects of the protocol. + + + One thing is certain: The addition of this capability to &yaz; should + have only a minimal impact on existing applications, and on the + interface to the software in general. Most likely, we will add an extra + layer of interface to the processing of EXPLAIN records, which will + convert back and forth between oident records (see + section Object Identifiers) and direct or + indirect references, given the current association setup. Implementations + based on any of the higher-level interfaces will most likely not have to + be changed at all. + + + + Summary and Synopsis + + + #include <comstack.h> + + #include <tcpip.h> /* this is for TCP/IP support */ + #include <xmosi.h> /* and this is for mOSI support */ + + COMSTACK cs_create(CS_TYPE type, int blocking, int protocol); + + COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking, + int protocol); + + int cs_bind(COMSTACK handle, int mode); + + int cs_connect(COMSTACK handle, void *address); + + int cs_rcvconnect(COMSTACK handle); + + int cs_listen(COMSTACK handle); + + COMSTACK cs_accept(COMSTACK handle); + + int cs_put(COMSTACK handle, char *buf, int len); + + int cs_get(COMSTACK handle, char **buf, int *size); + + int cs_more(COMSTACK handle); + + int cs_close(COMSTACK handle); + + int cs_look(COMSTACK handle); + + struct sockaddr_in *tcpip_strtoaddr(char *str); + + struct netbuf *mosi_strtoaddr(char *str); + + extern int cs_errno; + + void cs_perror(COMSTACK handle char *message); + + const char *cs_stackerr(COMSTACK handle); + + extern const char *cs_errlist[]; + + + + + + - - - -Introduction - - -The &comstack; -subsystem provides a transparent interface to different types of transport -stacks for the exchange of BER-encoded data. At present, the -RFC1729 method (BER over TCP/IP), and Peter Furniss' XTImOSI -stack are supported, but others may be added in time. The philosophy of the -module is to provide a simple interface by hiding unused options and -facilities of the underlying libraries. This is always done at the risk -of losing generality, and it may prove that the interface will need -extension later on. - - - -The interface is implemented in such a fashion that only the -sub-layers constructed to the transport methods that you wish to -use in your application are linked in. - - - -You will note that even though simplicity was a goal in the design, -the interface is still orders of magnitudes more complex than the -transport systems found in many other packages. One reason is that -the interface needs to support the somewhat different requirements of -the different lower-layer communications stacks; another important reason is -that the interface seeks to provide a more or less industrial-strength -approach to asynchronous event-handling. When no function is allowed -to block, things get more complex - particularly on the server -side. We urge you to have a look at the demonstration client and server -provided with the package. They are meant to be easily readable and -instructive, while still being at least moderately useful. - - - -Common Functions - -Managing Endpoints - - - COMSTACK cs_create(CS_TYPE type, int blocking, int protocol); - - - -Creates an instance of the protocol stack - a communications endpoint. -The type parameter determines the mode of communication. -At present, the values -tcpip_type -and -mosi_type -are recognized. The function returns a null-pointer if a system error -occurs. The blocking parameter should be one if you wish -the association to operate in blocking mode, zero otherwise. The -protocol field should be one of -PROTO_SR or PROTO_Z3950. - - - - -int cs_close(COMSTACK handle); - - - -Closes the connection (as elegantly as the lower layers will permit), -and releases the resouces pointed to by the -handle -parameter. The -handle -should not be referenced again after this call. - - - - -We really need a soft disconnect, don't we? - - - - -Data Exchange - - - (COMSTACK handle, char *buf, int len); - - - -Sends -buf -down the wire. In blocking mode, this function will return only when a -full buffer has been written, or an error has occurred. In nonblocking -mode, it's possible that the function will be unable to send the full -buffer at once, which will be indicated by a return value of 1. The -function will keep track of the number of octets already written; you -should call it repeatedly with the same values of buf -and len, until the buffer has been transmitted. -When a full buffer has been sent, the function will return 0 for -success. -1 indicates an error condition (see below). - - - - int cs_get(COMSTACK handle, char **buf, int *size); - - - -Receives a PDU from the peer. Returns the number of bytes -read. In nonblocking mode, it is possible that not all of the packet can be -read at once. In this case, the function returns 1. To simplify the -interface, the function is -responsible for managing the size of the buffer. It will be reallocated -if necessary to contain large packages, and will sometimes be moved -around internally by the subsystem when partial packages are read. Before -calling -cs_get -for the fist time, the buffer can be initialized to the null pointer, -and the length should also be set to 0 - cs_get will perform a -malloc(2) -on the buffer for you. When a full buffer has been read, the size of -the package is returned (which will always be greater than 1). -1 -indicates an error condition. - - - -See also the cs_more() function below. - - - - int cs_more(COMSTACK handle); - - - -The cs_more() function should be used in conjunction -with cs_get and -select(2). -The cs_get() function will sometimes -(notably in the TCP/IP mode) read more than a single protocol package -off the network. When this happens, the extra package is stored -by the subsystem. After callig cs_get(), and before -waiting for more input, You should always call -cs_more() -to check if there's a full protocol package already read. If -cs_more() -returns 1, -cs_get() -can be used to immediately fetch the new package. For the -mOSI -subsystem, the function should always return 0, but if you want your -stuff to be protocol independent, you should use it. - - - - -The cs_more() -function is required because the RFC1729-method -does not provide a way of separating individual PDUs, short of -partially decoding the BER. Some other implementations will carefully -nibble at the packet by calling -read(2) -several times. This was felt to be too inefficient (or at least -clumsy) - hence the call for this extra function. - - - - - int cs_look(COMSTACK handle); - - - -This function is useful when you're operating in nonblocking -mode. Call it when -select(2) -tells you there's something happening on the line. It returns one of -the following values: - - - -CS_NONE -No event is pending. The data found on the line was not a complete package. - - -CS_CONNECT -A response to your connect request has been received. Call -cs_rcvconnect -to process the event and to finalize the connection establishment. - - -CS_DISCON -The other side has closed the connection (or maybe sent a disconnect -request - but do we care? Maybe later). Call -cs_close to close your end of the association as well. - - -CS_LISTEN -A connect request has been received. Call cs_listen -to process the event. - - -CS_DATA -There's data to be found on the line. Call cs_get -to get it. - - - - - -You should be aware that even if -cs_look() -tells you that there's an event event pending, the corresponding -function may still return and tell you there was nothing to be found. -This means that only part of a package was available for reading. The -same event will show up again, when more data has arrived. - - - - - int cs_fileno(COMSTACK h); - - - -Returns the file descriptor of the association. Use this when -file-level operations on the endpoint are required -(select(2) operations, specifically). - - - - - -Client Side - - - int cs_connect(COMSTACK handle, void *address); - - - -Initiate a connection with the target at address -(more onaddresses below). The function will return 0 on success, and 1 if -the operation does not complete immediately (this will only -happen on a nonblocking endpoint). In this case, use -cs_rcvconnect to complete the operation, -when select(2) reports input pending on the -association. - - - - int cs_rcvconnect(COMSTACK handle); - - - -Complete a connect operation initiated by cs_connect(). -It will return 0 on success; 1 if the operation has not yet completed (in -this case, call the function again later); -1 if an error has occured. - - - - -Server Side - - -To establish a server under the inetd server, you -can use - - - - COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking, - int protocol); - - - -The socket parameter is an established socket (when -your application is invoked from inetd, the -socket will typically be 0. -The following parameters are identical to the ones for -cs_create. - - - - int cs_bind(COMSTACK handle, void *address, int mode) - - - -Binds a local address to the endpoint. Read about addresses below. The -mode parameter should be either -CS_CLIENT or CS_SERVER. - - - - int cs_listen(COMSTACK handle, char *addr, int *addrlen); - - - -Call this to process incoming events on an endpoint that has been -bound in listening mode. It will return 0 to indicate that the connect -request has been received, 1 to signal a partial reception, and -1 to -indicate an error condition. - - - - COMSTACK cs_accept(COMSTACK handle); - - - -This finalises the server-side association establishment, after -cs_listen has completed successfully. It returns a new connection -endpoint, which represents the new association. The application will -typically wish to fork off a process to handle the association at this -point, and continue listen for new connections on the old -handle. - - - -You can use the call - - - - char *cs_addrstr(COMSTACK); - - - -on an established connection to retrieve the hostname of the remote host. - - - -You may need to use this function with some care if your -name server service is slow or unreliable - - - - -Addresses - - -The low-level format of the addresses are different depending on the -mode of communication you have chosen. A function is provided by each -of the lower layers to map a user-friendly string-form address to the -binary form required by the lower layers. - - - - struct sockaddr_in *tcpip_strtoaddr(char *str); - - struct netbuf *mosi_strtoaddr(char *str); - - - -The format for TCP/IP addresses is straightforward: - - - -<host> [ ':' <portnum> ] - - - -The hostname can be either a domain name or an IP address. -The port number, if omitted, defaults to 210. - - - -For OSI, the format is - - - -[ <t-selector> '/' ] <host> [ ':' <port> ] - - - -The transport selector is given as an even number of hex digits. - - - -You'll note that the address format for the OSI mode are just a subset -of full presentation addresses. We use presentation addresses because -xtimosi doesn't, in itself, allow access to the X.500 Directory -service. We use a limited form, because we haven't yet come across an -implementation that used more of the elements of a full p-address. It -is a fairly simple matter to add the rest of the elements to the -address format as needed, however: Xtimosi does -support the full P-address structure. - - - -In both transport modes, the special hostname "@" is mapped -to any local address (the manifest constant INADDR_ANY). -It is used to establish local listening endpoints in the server role. - - - -When a connection has been established, you can use - - - - char cs_addrstr(COMSTACK h); - - - -to retrieve the host name of the peer system. The function returns a pointer -to a static area, which is overwritten on the next call to the function. - - - - -We have left the issue of X.500 name-to-address mapping open, for the -moment. It would be a simple matter to provide a table-based mapping, -if desired. Alternately, we could use the X.500 client-function that -is provided with the ISODE (although this would defeat some of the -purpose of using ThinOSI in the first place. We have been told that it -should be within the realm of the possible to implement a lightweight -implementation of the necessary X.500 client capabilities on top of -ThinOSI. This would be the ideal solution, we feel. On the other hand, it -still remains to be seen just what role the Directory will play in a world -populated by ThinOSI and other pragmatic solutions. - - - - - -Diagnostics - - -All functions return -1 if an error occurs. Typically, the functions -will return 0 on success, but the data exchange functions -(cs_get, cs_put, -cs_more) follow special rules. Consult their -descriptions. - - - -When a function (including the data exchange functions) reports an -error condition, use the function -cs_errno() to determine the cause of the -problem. The function - - - - void cs_perror(COMSTACK handle char *message); - - - -works like perror(2) and prints the -message argument, along with a system message, to -stderr. Use the character array - - - - extern const char *cs_errlist[]; - - - -to get hold of the message, if you want to process it differently. -The function - - - - const char *cs_stackerr(COMSTACK handle); - - - -Returns an error message from the lower layer, if one has been -provided. - - - -Enabling OSI Communication - -Installing Xtimosi - -Although you will have to download Peter Furniss' XTI/mOSI -implementation for yourself, we've tried to make the integration as -simple as possible. - - - -The latest version of xtimosi will generally be under - - - -ftp://pluto.ulcc.ac.uk/ulcc/thinosi/xtimosi/ - - - -When you have downloaded and unpacked the archive, it will (we assume) -have created a directory called xtimosi. -We suggest that you place this directory in the same -directory where you unpacked the &yaz; -distribution. This way, you shouldn't have to fiddle with the -makefiles of &yaz; beyond uncommenting a few lines. - - - -Go to xtimosi/src, and type "make libmosi.a/". -This should generally create the library, ready to use. - - - - -The currently available release of xtimosi has some inherent -problems that make it disfunction on certain platforms - eg. the -Digital OSF/1 workstations. It is supposedly primarily a -compiler problem, and we hope to see a release that is generally -portable. While we can't guarantee that it can be brought to work -on your platform, we'll be happy to talk to you about problems -that you might see, and relay information to the author of the -software. There are some signs that the gcc -compiler is more likely to produce a fully functional library, but this -hasn't been verified (we think that the problem is limited to the use -of hexadecimal escape-codes used in strings, which are silently -ignored by some compilers). - - -A problem has been encountered in the communication with -ISODE-based applications. If the ISODE presentation-user calls -PReadRequest() with a timeout value different -from OK or NOTOK, -he will get an immediate TIMEOUT abort when receiving large (>2041 -bytes, which is the SPDU-size that the ISODE likes to work with) packages -from an xtimosi-based implementation (probably most -other implementations as well, in fact). It seems to be a flaw in the -ISODE API, and the workaround (for ISODE users) is to either not -use an explicit timeout (switching to either blocking or -nonblocking mode), or to check that the timer really has expired -before closing the connection. - - - - -The next step in the installation is to modify the makefile in the toplevel -&yaz; -directory. The place to change is in the top of the file, and is -clearly marked with a comment. - - - -Now run make in the &yaz; toplevel directory (do a -make clean first, if the system has been previously -made without OSI support). Use the &yaz; -yaz-ztest and yaz-client -demo programs to verify that OSI communication works OK. Then, you can go -ahead and try to talk to other implementations. - - - - -Our interoperability experience is limited to version -7 of the Nordic SR-Nett package, which has had several -protocol errors fixed from the earlier releases. If you have -problems or successes in interoperating with other -implementations, we'd be glad to hear about it, or to help -you make things work, as our resources allow. - - - - -If you write your own applications based on &yaz;, and you wish to -include OSI support, the procedure is equally simple. You should -include the xmosi.h header file in addition to -comstack.h. xmosi.h -will define the manifest constant mosi_type, which you -should pass to the cs_create() function. In -addition, you should use the function mosi_strtoaddr() -rather than tcpip_strtoaddr() when you need to -prepare an address. - - - -When you link your application, you should include (after the -libyaz.a library) the libmosi.a -library, and the librfc.a library provided with -&yaz; (for OSI transport). - - -As always, it can be very useful, if not essential, to have a look at the -example applications to see how things are done. - - - -OSI Transport - - -Xtimosi requires an implementation of the OSI transport service under -the X/OPEN XTI API. We provide an implementation of the RFC1006 -encapsulation of OSI/TP0 in TCP/IP (through the Berkeley Sockets API), -as an independent part of &yaz; (it's found under the -rfc1006 directory). -If you have access to an OSI transport provider under XTI, -you should be able to make that work too, although it may require -tinkering with the mosi_strtoaddr() function. - - - -Presentation Context Management - - -To simplify the implementation, we use Peter Furniss' alternative (PRF) -option format -for the Control of the presentation negotiation phase. This format -is enabled by default when you -compile xtimosi. - - - -The current version of &yaz; does not support -presentation-layer negotiation of response record formats. The primary -reason is that we have had access to no other SR or Z39.50 -implementations over OSI that used this -method. Secondarily, we believe that the EXPLAIN facility is a superior -mechanism for relaying target capabilities in this respect. This is not to -say that we have no intentions of supporting presentation context -negotiation - we have just hitherto given it a lower priority than other -aspects of the protocol. - - -One thing is certain: The addition of this capability to &yaz; should -have only a minimal impact on existing applications, and on the -interface to the software in general. Most likely, we will add an extra -layer of interface to the processing of EXPLAIN records, which will -convert back and forth between oident records (see -section Object Identifiers) and direct or -indirect references, given the current association setup. Implementations -based on any of the higher-level interfaces will most likely not have to -be changed at all. - - - -Summary and Synopsis - - -#include <comstack.h> - -#include <tcpip.h> /* this is for TCP/IP support */ -#include <xmosi.h> /* and this is for mOSI support */ - -COMSTACK cs_create(CS_TYPE type, int blocking, int protocol); - -COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking, - int protocol); - -int cs_bind(COMSTACK handle, int mode); - -int cs_connect(COMSTACK handle, void *address); - -int cs_rcvconnect(COMSTACK handle); - -int cs_listen(COMSTACK handle); - -COMSTACK cs_accept(COMSTACK handle); - -int cs_put(COMSTACK handle, char *buf, int len); - -int cs_get(COMSTACK handle, char **buf, int *size); - -int cs_more(COMSTACK handle); - -int cs_close(COMSTACK handle); - -int cs_look(COMSTACK handle); - -struct sockaddr_in *tcpip_strtoaddr(char *str); - -struct netbuf *mosi_strtoaddr(char *str); - -extern int cs_errno; - -void cs_perror(COMSTACK handle char *message); - -const char *cs_stackerr(COMSTACK handle); - -extern const char *cs_errlist[]; - - - - - - diff --git a/doc/frontend.xml b/doc/frontend.xml index 61d85af..bc6f978 100644 --- a/doc/frontend.xml +++ b/doc/frontend.xml @@ -1,314 +1,320 @@ - -Making an IR Server for Your Database - -Introduction - - -If you aren't into documentation, a good way to learn how the -backend interface works is to look at the backend.h -file. Then, look at the small dummy-server in -ztest/ztest.c. Finally, you can have a look at -the seshigh.c file, which is where most of the -logic of the frontend server is located. The backend.h -file also makes a good reference, once you've chewed your way through -the prose of this file. - - - -If you have a database system that you would like to make available by -means of Z39.50, &yaz; basically offers your two options. You -can use the APIs provided by the &asn;, &odr;, and &comstack; -modules to -create and decode PDUs, and exchange them with a client. -Using this low-level interface gives you access to all fields and -options of the protocol, and you can construct your server as close -to your existing database as you like. -It is also a fairly involved process, requiring -you to set up an event-handling mechanism, protocol state machine, -etc. To simplify server implementation, we have implemented a compact -and simple, but reasonably full-functioned server-frontend that will -handle most of the protocol mechanics, while leaving you to -concentrate on your database interface. - - - - -The backend interface was designed in anticipation of a specific -integration task, while still attempting to achieve some degree of -generality. We realise fully that there are points where the -interface can be improved significantly. If you have specific -functions or parameters that you think could be useful, send us a -mail (or better, sign on to the mailing list referred to in the -toplevel README file). We will try to fit good suggestions into future -releases, to the extent that it can be done without requiring -too many structural changes in existing applications. - - - - -The Database Frontend - - -We refer to this software as a generic database frontend. Your -database system is the backend database, and the -interface between the two is called the backend API. -The backend API consists of a small number of function handlers and -structure definitions. You are required to provide the -main() routine for the server (which can be -quite simple), as well as a set of handlers to match each of the prototypes. -The interface functions that you write can use any mechanism you like -to communicate with your database system: You might link the whole -thing together with your database application and access it by -function calls; you might use IPC to talk to a database server -somewhere; or you might link with third-party software that handles -the communication for you (like a commercial database client library). -At any rate, the handlers will perform the tasks of: - - - - - -Initialization. - - - -Searching. - - - -Fetching records. - - - -Scanning the database index (optional - if you wish to implement SCAN). - - - -Extended Services (optional). - - - -Result-Set Delete (optional). - - - -Result-Set Sort (optional). - - - - - -(more functions will be added in time to support as much of -Z39.50-1995 as possible). - - - -The Backend API - - -The headers files that you need to use the interface are in the -include/yaz directory. They are called -statserv.h and backend.h. They -will include other files from the include/yaz -directory, so you'll probably want to use the -I option of your -compiler to tell it where to find the files. When you run -make in the toplevel &yaz; directory, -everything you need to create your server is put the -lib/libyaz.a library. - - - -Your main() Routine - - -As mentioned, your main() routine can be quite brief. -If you want to initialize global parameters, or read global configuration -tables, this is the place to do it. At the end of the routine, you should -call the function - - - -int statserv_main(int argc, char **argv, - bend_initresult *(*bend_init)(bend_initrequest *r), - void (*bend_close)(void *handle)); - - - -The third and fourth arguments are pointers to handlers. Handler -bend_init is called whenever the server receives -an Initialize Request, so it serves as a Z39.50 session initializer. The -bend_close handler is called when the session is -closed. - - - -statserv_main will establish listening sockets -according to the parameters given. When connection requests are received, -the event handler will typically fork() and -create a sub-process to handle a new connection. -Alternatively the server may be setup to create threads for each connection. -If you do use global variables and forking, you should be aware, then, -that these cannot be shared between associations, unless you explicitly -disable forking by command line parameters. - - - -The server provides a mechanism for controlling some of its behavior -without using command-line options. The function - - - -statserv_options_block *statserv_getcontrol(void); - - - -Will return a pointer to a struct statserv_options_block -describing the current default settings of the server. The structure -contains these elements: - - -int dynamic -A boolean value, which determines whether the server -will fork on each incoming request (TRUE), or not (FALSE). Default is -TRUE. This flag is only read by UNIX-based servers (WIN32 based servers -doesn't fork). - - -int threads -A boolean value, which determines whether the server -will create a thread on each incoming request (TRUE), or not (FALSE). -Default is FALSE. This flag is only read by UNIX-based servers that offer -POSIX Threads support. WIN32-based servers always operate in threaded mode. - - -int inetd -A boolean value, which determines whether the server -will operates under a UNIX INET daemon (inetd). Default is FALSE. - - -int loglevel -Set this by ORing the constants defined in -include/yaz/yaz-log.h. - - -char logfile[ODR_MAXNAME+1] -File for diagnostic output ("": stderr). - -char apdufile[ODR_MAXNAME+1] - -Name of file for logging incoming and outgoing APDUs ("": don't -log APDUs, "-": stderr). - - -char default_listen[1024] -Same form as the command-line specification of -listener address. "": no default listener address. -Default is to listen at "tcp:@:9999". You can only -specify one default listener address in this fashion. - - -enum oid_proto default_proto; -Either PROTO_SR or -PROTO_Z3950. Default is PROTO_Z39_50. - -int idle_timeout; -Maximum session idletime, in minutes. Zero indicates -no (infinite) timeout. Default is 120 minutes. - - -int maxrecordsize; -Maximum permissible record (message) size. Default -is 1Mb. This amount of memory will only be allocated if a client requests a -very large amount of records in one operation (or a big record). Set it -to a lower number -if you are worried about resource consumption on your host system. - - -char configname[ODR_MAXNAME+1] -Passed to the backend when a new connection is received. - - -char setuid[ODR_MAXNAME+1] -Set user id to the user specified, after binding -the listener addresses. - - -void (*bend_start)(struct statserv_options_block *p) -Pointer to function which is called after the command line -options have been parsed - but before the server starts listening. For -forked UNIX servers this handler is called in the mother process; for -threaded servers this handler is called in the main thread. The default -value of this pointer is NULL in which case it isn't invoked by the frontend -server. When the server operates as an NT service this handler is called -whenever the service is started. - - -void (*bend_stop)(struct statserv_options_block *p) -Pointer to function which is called whenver the server -has stopped listening for incoming connections. This function pointer -has a default value of NULL in which case it isn't called. -When the server operates as an NT service this handler is called -whenever the service is stopped. - - -void *handle -User defined pointer (default value NULL). -This is a per-server handle that can be used to specify "user-data". -Do not confuse this with the session-handle as returned by bend_init. - - - - - - -The pointer returned by statserv_getcontrol points to -a static area. You are allowed to change the contents of the structure, -but the changes will not take effect before you call - - - -void statserv_setcontrol(statserv_options_block *block); - - - - -that you should generally update this structure before calling -statserv_main(). - - - - -The Backend Functions - - -For each service of the protocol, the backend interface declares one or -two functions. You are required to provide implementations of the -functions representing the services that you wish to implement. - - -Init - - -bend_initresult (*bend_init)(bend_initrequest *r); - - - -This handler is called once for each new connection request, after -a new process/thread has been created, and an Initialize Request has been -received from the client. The pointer to the bend_init -handler is passed in the call to statserv_start. - - -Unlike previous versions of YAZ, the bend_init also -serves as a handler that defines the Z39.50 services that the backend -wish to support. Pointers to all service handlers, -including search - and fetch must be specified here in this handler. - - -The request - and result structures are defined as - - - + + Making an IR Server for Your Database + + Introduction + + + If you aren't into documentation, a good way to learn how the + backend interface works is to look at the backend.h + file. Then, look at the small dummy-server in + ztest/ztest.c. Finally, you can have a look at + the seshigh.c file, which is where most of the + logic of the frontend server is located. The backend.h + file also makes a good reference, once you've chewed your way through + the prose of this file. + + + + If you have a database system that you would like to make available by + means of Z39.50, &yaz; basically offers your two options. You + can use the APIs provided by the &asn;, &odr;, and &comstack; + modules to + create and decode PDUs, and exchange them with a client. + Using this low-level interface gives you access to all fields and + options of the protocol, and you can construct your server as close + to your existing database as you like. + It is also a fairly involved process, requiring + you to set up an event-handling mechanism, protocol state machine, + etc. To simplify server implementation, we have implemented a compact + and simple, but reasonably full-functioned server-frontend that will + handle most of the protocol mechanics, while leaving you to + concentrate on your database interface. + + + + + The backend interface was designed in anticipation of a specific + integration task, while still attempting to achieve some degree of + generality. We realise fully that there are points where the + interface can be improved significantly. If you have specific + functions or parameters that you think could be useful, send us a + mail (or better, sign on to the mailing list referred to in the + toplevel README file). We will try to fit good suggestions into future + releases, to the extent that it can be done without requiring + too many structural changes in existing applications. + + + + + The Database Frontend + + + We refer to this software as a generic database frontend. Your + database system is the backend database, and the + interface between the two is called the backend API. + The backend API consists of a small number of function handlers and + structure definitions. You are required to provide the + main() routine for the server (which can be + quite simple), as well as a set of handlers to match each of the prototypes. + The interface functions that you write can use any mechanism you like + to communicate with your database system: You might link the whole + thing together with your database application and access it by + function calls; you might use IPC to talk to a database server + somewhere; or you might link with third-party software that handles + the communication for you (like a commercial database client library). + At any rate, the handlers will perform the tasks of: + + + + + + Initialization. + + + + Searching. + + + + Fetching records. + + + + Scanning the database index (optional - if you wish to implement SCAN). + + + + Extended Services (optional). + + + + Result-Set Delete (optional). + + + + Result-Set Sort (optional). + + + + + + (more functions will be added in time to support as much of + Z39.50-1995 as possible). + + + + The Backend API + + + The headers files that you need to use the interface are in the + include/yaz directory. They are called + statserv.h and backend.h. They + will include other files from the include/yaz + directory, so you'll probably want to use the -I option of your + compiler to tell it where to find the files. When you run + make in the toplevel &yaz; directory, + everything you need to create your server is put the + lib/libyaz.a library. + + + + Your main() Routine + + + As mentioned, your main() routine can be quite brief. + If you want to initialize global parameters, or read global configuration + tables, this is the place to do it. At the end of the routine, you should + call the function + + + + int statserv_main(int argc, char **argv, + bend_initresult *(*bend_init)(bend_initrequest *r), + void (*bend_close)(void *handle)); + + + + The third and fourth arguments are pointers to handlers. Handler + bend_init is called whenever the server receives + an Initialize Request, so it serves as a Z39.50 session initializer. The + bend_close handler is called when the session is + closed. + + + + statserv_main will establish listening sockets + according to the parameters given. When connection requests are received, + the event handler will typically fork() and + create a sub-process to handle a new connection. + Alternatively the server may be setup to create threads for each connection. + If you do use global variables and forking, you should be aware, then, + that these cannot be shared between associations, unless you explicitly + disable forking by command line parameters. + + + + The server provides a mechanism for controlling some of its behavior + without using command-line options. The function + + + + statserv_options_block *statserv_getcontrol(void); + + + + Will return a pointer to a struct statserv_options_block + describing the current default settings of the server. The structure + contains these elements: + + + int dynamic + A boolean value, which determines whether the server + will fork on each incoming request (TRUE), or not (FALSE). Default is + TRUE. This flag is only read by UNIX-based servers (WIN32 based servers + doesn't fork). + + + int threads + A boolean value, which determines whether the server + will create a thread on each incoming request (TRUE), or not (FALSE). + Default is FALSE. This flag is only read by UNIX-based servers that offer + POSIX Threads support. WIN32-based servers always operate in threaded mode. + + + int inetd + A boolean value, which determines whether the server + will operates under a UNIX INET daemon (inetd). Default is FALSE. + + + int loglevel + Set this by ORing the constants defined in + include/yaz/yaz-log.h. + + + char logfile[ODR_MAXNAME+1] + File for diagnostic output ("": stderr). + + char apdufile[ODR_MAXNAME+1] + + Name of file for logging incoming and outgoing APDUs ("": don't + log APDUs, "-": stderr). + + + char default_listen[1024] + Same form as the command-line specification of + listener address. "": no default listener address. + Default is to listen at "tcp:@:9999". You can only + specify one default listener address in this fashion. + + + enum oid_proto default_proto; + Either PROTO_SR or + PROTO_Z3950. Default is PROTO_Z39_50. + + int idle_timeout; + Maximum session idletime, in minutes. Zero indicates + no (infinite) timeout. Default is 120 minutes. + + + int maxrecordsize; + Maximum permissible record (message) size. Default + is 1Mb. This amount of memory will only be allocated if a client requests a + very large amount of records in one operation (or a big record). Set it + to a lower number + if you are worried about resource consumption on your host system. + + + char configname[ODR_MAXNAME+1] + Passed to the backend when a new connection is received. + + + char setuid[ODR_MAXNAME+1] + Set user id to the user specified, after binding + the listener addresses. + + + + void (*bend_start)(struct statserv_options_block *p) + Pointer to function which is called after the + command line options have been parsed - but before the server + starts listening. + For forked UNIX servers this handler is called in the mother + process; for threaded servers this handler is called in the + main thread. + The default value of this pointer is NULL in which case it + isn't invoked by the frontend server. + When the server operates as an NT service this handler is called + whenever the service is started. + + + + void (*bend_stop)(struct statserv_options_block *p) + Pointer to function which is called whenver the server + has stopped listening for incoming connections. This function pointer + has a default value of NULL in which case it isn't called. + When the server operates as an NT service this handler is called + whenever the service is stopped. + + + void *handle + User defined pointer (default value NULL). + This is a per-server handle that can be used to specify "user-data". + Do not confuse this with the session-handle as returned by bend_init. + + + + + + + The pointer returned by statserv_getcontrol points to + a static area. You are allowed to change the contents of the structure, + but the changes will not take effect before you call + + + + void statserv_setcontrol(statserv_options_block *block); + + + + + that you should generally update this structure before calling + statserv_main(). + + + + + The Backend Functions + + + For each service of the protocol, the backend interface declares one or + two functions. You are required to provide implementations of the + functions representing the services that you wish to implement. + + + Init + + + bend_initresult (*bend_init)(bend_initrequest *r); + + + + This handler is called once for each new connection request, after + a new process/thread has been created, and an Initialize Request has + been received from the client. The pointer to the + bend_init handler is passed in the call to + statserv_start. + + + Unlike previous versions of YAZ, the bend_init also + serves as a handler that defines the Z39.50 services that the backend + wish to support. Pointers to all service handlers, + including search - and fetch must be specified here in this handler. + + + The request - and result structures are defined as + + + typedef struct bend_initrequest { Z_IdAuthentication *auth; @@ -335,63 +341,63 @@ typedef struct bend_initresult char *errstring; /* system error string or NULL */ void *handle; /* private handle to the backend module */ } bend_initresult; - - - -In general, the server frontend expects that the -bend_*result pointer that you return is valid at -least until the next call to a bend_* function. -This applies to all of the functions described herein. The parameter -structure passed to you in the call belongs to the server frontend, and -you should not make assumptions about its contents after the current -function call has completed. In other words, if you want to retain any -of the contents of a request structure, you should copy them. - - - -The errcode should be zero if the initialization of -the backend went well. Any other value will be interpreted as an error. -The errstring isn't used in the current version, but one -optin would be to stick it in the initResponse as a VisibleString. -The handle is the most important parameter. It should -be set to some value that uniquely identifies the current session to -the backend implementation. It is used by the frontend server in any -future calls to a backend function. -The typical use is to set it to point to a dynamically allocated state -structure that is private to your backend module. - - - -The auth member holds the authentication information -part of the Z39.50 Initialize Request. Interpret this if your serves -requires authentication. - - - -The members peer_name, -implementation_name and -implementation_version holds DNS of client, name -of client (Z39.50) implementation - and version. - - - -The bend_ - members are set to NULL when -bend_init is called. Modify the pointers by setting them -to point to backend functions. - - - - -Search and retrieve - -We now describe the handlers that are required to support search - -and retrieve. You must support two functions - one for seearch - and one -for fetch (retrieval of one record). If desirable you can provide a -third handler which is called when a present request is received which -allows you to optimize retrieval of multiple-records. - - - + + + + In general, the server frontend expects that the + bend_*result pointer that you return is valid at + least until the next call to a bend_* function. + This applies to all of the functions described herein. The parameter + structure passed to you in the call belongs to the server frontend, and + you should not make assumptions about its contents after the current + function call has completed. In other words, if you want to retain any + of the contents of a request structure, you should copy them. + + + + The errcode should be zero if the initialization of + the backend went well. Any other value will be interpreted as an error. + The errstring isn't used in the current version, but + one option would be to stick it in the initResponse as a VisibleString. + The handle is the most important parameter. It should + be set to some value that uniquely identifies the current session to + the backend implementation. It is used by the frontend server in any + future calls to a backend function. + The typical use is to set it to point to a dynamically allocated state + structure that is private to your backend module. + + + + The auth member holds the authentication information + part of the Z39.50 Initialize Request. Interpret this if your serves + requires authentication. + + + + The members peer_name, + implementation_name and + implementation_version holds DNS of client, name + of client (Z39.50) implementation - and version. + + + + The bend_ - members are set to NULL when + bend_init is called. Modify the pointers by + setting them to point to backend functions. + + + + + Search and retrieve + + We now describe the handlers that are required to support search - + and retrieve. You must support two functions - one for seearch - and one + for fetch (retrieval of one record). If desirable you can provide a + third handler which is called when a present request is received which + allows you to optimize retrieval of multiple-records. + + + int (*bend_search) (void *handle, bend_search_rr *rr); typedef struct { @@ -413,43 +419,43 @@ typedef struct { char *errstring; /* system error string or NULL */ } bend_search_rr; - - - -The bend_search handler is a fairly close -approximation of a protocol Search Request - and Response PDUs -The setname is the resultSetName from the protocol. -You are required to establish a mapping between the set name and whatever -your backend database likes to use. -Similarly, the replace_set is a boolean value -corresponding to the resultSetIndicator field in the protocol. -num_bases/basenames is a length of/array of character -pointers to the database names provided by the client. -The query is the full query structure as defined in the -protocol ASN.1 specification. -It can be either of the possible query types, and it's up to you to -determine if you can handle the provided query type. -Rather than reproduce the C interface here, we'll refer you to the -structure definitions in the file -include/yaz/z-core.h. If you want to look at the -attributeSetId OID of the RPN query, you can either match it against -your own internal tables, or you can use the -oid_getentbyoid function provided by &yaz;. - - - -The structure contains a number of hits, and an -errcode/errstring pair. If an error occurs -during the search, or if you're unhappy with the request, you should -set the errcode to a value from the BIB-1 diagnostic set. The value -will then be returned to the user in a nonsurrogate diagnostic record -in the response. The errstring, if provided, will -go in the addinfo field. Look at the protocol definition for the -defined error codes, and the suggested uses of the addinfo field. - - - - + + + + The bend_search handler is a fairly close + approximation of a protocol Search Request - and Response PDUs + The setname is the resultSetName from the protocol. + You are required to establish a mapping between the set name and whatever + your backend database likes to use. + Similarly, the replace_set is a boolean value + corresponding to the resultSetIndicator field in the protocol. + num_bases/basenames is a length of/array of character + pointers to the database names provided by the client. + The query is the full query structure as defined in the + protocol ASN.1 specification. + It can be either of the possible query types, and it's up to you to + determine if you can handle the provided query type. + Rather than reproduce the C interface here, we'll refer you to the + structure definitions in the file + include/yaz/z-core.h. If you want to look at the + attributeSetId OID of the RPN query, you can either match it against + your own internal tables, or you can use the + oid_getentbyoid function provided by &yaz;. + + + + The structure contains a number of hits, and an + errcode/errstring pair. If an error occurs + during the search, or if you're unhappy with the request, you should + set the errcode to a value from the BIB-1 diagnostic set. The value + will then be returned to the user in a nonsurrogate diagnostic record + in the response. The errstring, if provided, will + go in the addinfo field. Look at the protocol definition for the + defined error codes, and the suggested uses of the addinfo field. + + + + int (*bend_fetch) (void *handle, bend_fetch_rr *rr); typedef struct bend_fetch_rr { @@ -472,67 +478,71 @@ typedef struct bend_fetch_rr { char *errstring; /* system error string or NULL */ int surrogate_flag; /* surrogate diagnostic */ } bend_fetch_rr; - - - -The frontend server calls the bend_fetch handler when -it needs database records to fulfill a Search Request or a Present Request. -The setname is simply the name of the result set -that holds the reference to the desired record. -The number is the offset into the set (with 1 -being the first record in the set). The format field -is the record format requested by the client (See section -Object Identifiers). The value -VAL_NONE indicates that the client did not -request a specific format. The stream argument -is an &odr; stream which should be used for -allocating space for structured data records. The stream will be reset when -all records have been assembled, and the response package has been transmitted. -For unstructured data, the backend is responsible for maintaining a static -or dynamic buffer for the record between calls. - - - -In the structure, the basename is the name of the -database that holds the -record. len is the length of the record returned, in -bytes, and record is a pointer to the record. -Last_in_set should be nonzero only if the record -returned is the last one in the given result set. errcode -and errstring, if given, will be -interpreted as a global error pertaining to the set, and will be returned -in a non-surrogate-diagnostic. If you wish to return the error as a -surrogate-diagnostic (local error) you can do this by setting -surrogate_flag to 1 also. - - - -If the len field has the value -1, then -record is assumed to point to a constructed data -type. The format field will be used to determine -which encoder should be used to serialize the data. - - - - -If your backend generates structured records, it should use -odr_malloc() on the provided stream for allocating -data: This allows the frontend server to keep track of the record sizes. - - - - -The format field is mapped to an object identifier -in the direct reference of the resulting EXTERNAL representation of the record. - - - - -The current version of &yaz; only supports the direct reference mode. - - - - + + + + The frontend server calls the bend_fetch handler + when it needs database records to fulfill a Search Request or a Present + Request. + The setname is simply the name of the result set + that holds the reference to the desired record. + The number is the offset into the set (with 1 + being the first record in the set). The format field + is the record format requested by the client (See section + Object Identifiers). The value + VAL_NONE indicates that the client did not + request a specific format. The stream argument + is an &odr; stream which should be used for + allocating space for structured data records. + The stream will be reset when all records have been assembled, and + the response package has been transmitted. + For unstructured data, the backend is responsible for maintaining a static + or dynamic buffer for the record between calls. + + + + In the structure, the basename is the name of the + database that holds the + record. len is the length of the record returned, in + bytes, and record is a pointer to the record. + Last_in_set should be nonzero only if the record + returned is the last one in the given result set. + errcode and errstring, if + given, will be interpreted as a global error pertaining to the + set, and will be returned in a non-surrogate-diagnostic. + If you wish to return the error as a surrogate-diagnostic + (local error) you can do this by setting + surrogate_flag to 1 also. + + + + If the len field has the value -1, then + record is assumed to point to a constructed data + type. The format field will be used to determine + which encoder should be used to serialize the data. + + + + + If your backend generates structured records, it should use + odr_malloc() on the provided stream for allocating + data: This allows the frontend server to keep track of the record sizes. + + + + + The format field is mapped to an object identifier + in the direct reference of the resulting EXTERNAL representation + of the record. + + + + + The current version of &yaz; only supports the direct reference mode. + + + + int (*bend_present) (void *handle, bend_present_rr *rr); typedef struct { @@ -551,33 +561,33 @@ typedef struct { int errcode; /* 0==OK */ char *errstring; /* system error string or NULL */ } bend_present_rr; - - - -The bend_present handler is called when -the server receives a Present Request. The setname, -start and number is the -name of the result set - start position - and number of records to -be retrieved respectively. format and -comp is the preferred transfer syntax and element -specifications of the present request. - - -Note that this is handler serves as a supplement for -bend_fetch and need not to be defined in order to -support search - and retrieve. - - - - -Delete - - -For backends that supports delete of a result set only one handler -must be defined. - - - + + + + The bend_present handler is called when + the server receives a Present Request. The setname, + start and number is the + name of the result set - start position - and number of records to + be retrieved respectively. format and + comp is the preferred transfer syntax and element + specifications of the present request. + + + Note that this is handler serves as a supplement for + bend_fetch and need not to be defined in order to + support search - and retrieve. + + + + + Delete + + + For backends that supports delete of a result set only one handler + must be defined. + + + int (*bend_delete)(void *handle, bend_delete_rr *rr); typedef struct bend_delete_rr { @@ -590,28 +600,28 @@ typedef struct bend_delete_rr { ODR stream; ODR print; } bend_delete_rr; - + - - -The delete set function definition is rather primitive, mostly because we -have had no practical need for it as of yet. If someone wants -to provide a full delete service, we'd be happy to add the -extra parameters that are required. Are there clients out there -that will actually delete sets they no longer need? - - + + + The delete set function definition is rather primitive, mostly because we + have had no practical need for it as of yet. If someone wants + to provide a full delete service, we'd be happy to add the + extra parameters that are required. Are there clients out there + that will actually delete sets they no longer need? + + - + -scan + scan - -For servers that wish to offer the scan service one handler -must be defined. - + + For servers that wish to offer the scan service one handler + must be defined. + - + int (*bend_delete)(void *handle, bend_delete_rr *rr); typedef enum { @@ -637,172 +647,188 @@ typedef struct bend_scan_rr { int errcode; char *errstring; } bend_scan_rr; - - - - -Application Invocation - - -The finished application has the following -invocation syntax (by way of statserv_main()): - - - -appname [-szSiTu -a apdufile -l logfile -v loglevel -c config] -[listener ...] - - - -The options are - - - --a file - -Specify a file for dumping PDUs (for diagnostic purposes). -The special name "-" sends output to stderr. - - --S - -Don't fork or make threads on connection requests. This is good for -debugging, but not recommended for real operation: Although the server is -asynchronous and non-blocking, it can be nice to keep a software -malfunction (okay then, a crash) from affecting all current users. - - --T - -Operate the server in threaded mode. The server creates a thread -for each connection rather than a fork a process. Only available -on UNIX systems that offers POSIX threads. - - --s - -Use the SR protocol (obsolete). - - --z - -Use the Z39.50 protocol (default). These two options complement -each other. You can use both multiple times on the same command -line, between listener-specifications (see below). This way, you -can set up the server to listen for connections in both protocols -concurrently, on different local ports. - - --l file -The logfile. - - --c config -A user option that serves as a specifier for some -sort of configuration, e.g. a filename. -The argument to this option is transferred to member -confignameof the statserv_options_block. - - --v level - -The log level. Use a comma-separated list of members of the set -{fatal,debug,warn,log,all,none}. - - --u userid - -Set user ID. Sets the real UID of the server process to that of the -given user. It's useful if you aren't comfortable with having the -server run as root, but you need to start it as such to bind a -privileged port. - - --w dir - -Working directory. - - --i - -Use this when running from the inetd server. - - --t minutes - -Idle session timeout, in minutes. - - --k size - -Maximum record size/message size, in kilobytes. - - - - - - - -A listener specification consists of a transport mode followed by a -colon (:) followed by a listener address. The transport mode is -either osi or tcp. - - - -For TCP, an address has the form - - - - hostname | IP-number [: portnumber] - - - -The port number defaults to 210 (standard Z39.50 port). - - - -For osi, the address form is - - - - [t-selector /] hostname | IP-number [: portnumber] - - - -The transport selector is given as a string of hex digits (with an even -number of digits). The default port number is 102 (RFC1006 port). - - - -Examples - - - - tcp:dranet.dra.com - - osi:0402/dbserver.osiworld.com:3000 - - - -In both cases, the special hostname "@" is mapped to -the address INADDR_ANY, which causes the server to listen on any local -interface. To start the server listening on the registered ports for -Z39.50 and SR over OSI/RFC1006, and to drop root privileges once the -ports are bound, execute the server like this (from a root shell): - - - - my-server -u daemon tcp:@ -s osi:@ - - - -You can replace daemon with another user, eg. your -own account, or a dedicated IR server account. -my-server should be the name of your -server application. You can test the procedure with the -yaz-ztest application. - - - - - + + + + + Application Invocation + + + The finished application has the following + invocation syntax (by way of statserv_main()): + + + + appname [-szSiTu -a apdufile -l logfile -v loglevel -c config] + [listener ...] + + + + The options are + + + + -a file + + Specify a file for dumping PDUs (for diagnostic purposes). + The special name "-" sends output to + stderr. + + + -S + + Don't fork or make threads on connection requests. This is good for + debugging, but not recommended for real operation: Although the + server is asynchronous and non-blocking, it can be nice to keep + a software malfunction (okay then, a crash) from affecting all + current users. + + + -T + + Operate the server in threaded mode. The server creates a thread + for each connection rather than a fork a process. Only available + on UNIX systems that offers POSIX threads. + + + -s + + Use the SR protocol (obsolete). + + + -z + + Use the Z39.50 protocol (default). These two options complement + each other. You can use both multiple times on the same command + line, between listener-specifications (see below). This way, you + can set up the server to listen for connections in both protocols + concurrently, on different local ports. + + + -l file + The logfile. + + + -c config + A user option that serves as a specifier for some + sort of configuration, e.g. a filename. + The argument to this option is transferred to member + confignameof the + statserv_options_block. + + + -v level + + The log level. Use a comma-separated list of members of the set + {fatal,debug,warn,log,all,none}. + + + -u userid + + Set user ID. Sets the real UID of the server process to that of the + given user. It's useful if you aren't comfortable with having the + server run as root, but you need to start it as such to bind a + privileged port. + + + -w dir + + Working directory. + + + -i + + Use this when running from the inetd server. + + + -t minutes + + Idle session timeout, in minutes. + + + -k size + + Maximum record size/message size, in kilobytes. + + + + + + + A listener specification consists of a transport mode followed by a + colon (:) followed by a listener address. The transport mode is + either osi or tcp. + + + + For TCP, an address has the form + + + + hostname | IP-number [: portnumber] + + + + The port number defaults to 210 (standard Z39.50 port). + + + + For osi, the address form is + + + + [t-selector /] hostname | IP-number [: portnumber] + + + + The transport selector is given as a string of hex digits (with an even + number of digits). The default port number is 102 (RFC1006 port). + + + + Examples + + + + tcp:dranet.dra.com + + osi:0402/dbserver.osiworld.com:3000 + + + + In both cases, the special hostname "@" is mapped to + the address INADDR_ANY, which causes the server to listen on any local + interface. To start the server listening on the registered ports for + Z39.50 and SR over OSI/RFC1006, and to drop root privileges once the + ports are bound, execute the server like this (from a root shell): + + + + my-server -u daemon tcp:@ -s osi:@ + + + + You can replace daemon with another user, eg. your + own account, or a dedicated IR server account. + my-server should be the name of your + server application. You can test the procedure with the + yaz-ztest application. + + + + + + diff --git a/doc/future.xml b/doc/future.xml index c40a3c1..391f35e 100644 --- a/doc/future.xml +++ b/doc/future.xml @@ -1,31 +1,46 @@ - -Future Directions + + Future Directions - -We have a new and better version of the frontend server on the drawing -board. Resources and external commitments will govern when we'll be -able to do something real with it. Fetures should include greater -flexibility, greter support for access/resource control, and easy -support for Explain (possibly with Zebra as an extra database engine). - + + We have a new and better version of the frontend server on the drawing + board. Resources and external commitments will govern when we'll be + able to do something real with it. Fetures should include greater + flexibility, greter support for access/resource control, and easy + support for Explain (possibly with Zebra as an extra database engine). + - -We now support all PDUs of Z39.50-1995. If there is one of the -supporting structures that you need but can't find in the prt*.h -files, send us a note; it may be on its way. - + + We now support all PDUs of Z39.50-1995. If there is one of the + supporting structures that you need but can't find in the prt*.h + files, send us a note; it may be on its way. + - -The 'retrieval' module needs to be finalized and documented. We think -it can form a useful resource for people dealing with complex record -structures, but for now, you'll mostly have to chew through the code -yourself to make use of it. Not acceptable. - + + The 'retrieval' module needs to be finalized and documented. We think + it can form a useful resource for people dealing with complex record + structures, but for now, you'll mostly have to chew through the code + yourself to make use of it. Not acceptable. + - -Other than that, YAZ generally moves in the directions which appear to -make the most people happy (including ourselves, as prime users of the -software). If there's something you'd like to see in here, then drop -us a note and let's see what we can come up with. - - \ No newline at end of file + + Other than that, YAZ generally moves in the directions which appear to + make the most people happy (including ourselves, as prime users of the + software). If there's something you'd like to see in here, then drop + us a note and let's see what we can come up with. + + + + diff --git a/doc/indexdata.xml b/doc/indexdata.xml index 7095474..29fd69b 100644 --- a/doc/indexdata.xml +++ b/doc/indexdata.xml @@ -1,49 +1,63 @@ - -About Index Data + + About Index Data - -Index Data is a consulting and software-development enterprise that -specialises in library and information management systems. Our -interests and expertise span a broad range of related fields, and one -of our primary, long-term objectives is the development of a powerful -information management -system with open network interfaces and hypermedia capabilities. - -We make this software available free of charge, on a fairly unrestrictive -license; as a service to the networking community, and to further the -development of quality software for open network communication. - -We'll be happy to answer questions about the software, and about ourselves -in general. - - -
-Index Data Aps -Købmagergade 43 -1150 Copenhagen K -Denmark -Phone +45 3341 0100 -Fax +45 3341 0101 -Email info@indexdata.dk -
+ + Index Data is a consulting and software-development enterprise that + specialises in library and information management systems. Our + interests and expertise span a broad range of related fields, and one + of our primary, long-term objectives is the development of a powerful + information management + system with open network interfaces and hypermedia capabilities. + + We make this software available free of charge, on a fairly unrestrictive + license; as a service to the networking community, and to further the + development of quality software for open network communication. + + We'll be happy to answer questions about the software, and about ourselves + in general. + + +
+ Index Data Aps + Købmagergade 43 + 1150 Copenhagen K + Denmark + Phone +45 3341 0100 + Fax +45 3341 0101 + Email info@indexdata.dk +
- - + + -The Hacker's Jargon File has the following to say about the -use of the -prefix "YA" in the name of a software product. - - - -Yet Another. adj. 1. Of your own work: A -humorous allusion often used in titles to acknowledge that the -topic is not original, though the content is. As in "Yet Another -AI Group" or "Yet Another Simulated Annealing Algorithm". -2. Of -others' work: Describes something of which there are already far -too many. - + The Hacker's Jargon File has the following to say about the + use of the + prefix "YA" in the name of a software product. + + + + Yet Another. adj. 1. Of your own work: A + humorous allusion often used in titles to acknowledge that the + topic is not original, though the content is. As in "Yet Another + AI Group" or "Yet Another Simulated Annealing Algorithm". + 2. Of + others' work: Describes something of which there are already far + too many. + - - + + + diff --git a/doc/installation.xml b/doc/installation.xml index cbd4abd..6d5c09f 100644 --- a/doc/installation.xml +++ b/doc/installation.xml @@ -1,267 +1,286 @@ - -Compilation and Installation - - -The latest version of the software will generally be found at - - - -http://ftp.indexdata.dk/pub/yaz/ - - -We have tried our best to keep the software portable, and on many -platforms, you should be able to compile everything with little or no changes. -So far, the software has been ported -to the following platforms with little or no difficulties. - - -Unix systems - -HP/UX -SunOS/Solaris -DEC Unix -Linux -IBM AIX -Data General DG/UX (with some CFLAGS tinkering) - -SGI/IRIX -DDE Supermax - -Non-unix systems - -Apple Macintosh (using the Codewarrior programming -environment and the GUSI socket libraries) -MS Windows 95/98/NT/W2K (Win32) -IBM AS/400 - - - - - -If you move the software to other platforms, we'd be grateful if you'd -let us know about it. If you run into difficulties, we will try to help if we -can, and if you solve the problems, we would be happy to -include your fixes in the next release. So far, we have mostly avoided -#ifdefs for individual platforms, and we'd like to keep it that -way as far as it makes sense. - - - -We maintain a mailing-list for the purpose of announcing new releases and -bug-fixes, as well as general discussion. Subscribe by sending mail to -yaz-request@indexdata.dk. -General questions and problems can be directed at -yaz-help@indexdata.dk, or -the address given at the top of this document. - - -UNIX - - -Note that if your system doesn't have a native ANSI C compiler, you may -have to acquire one separately. We recommend gcc. - - -For UNIX we use GNU configure to create Makefiles for &yaz;. -Generally it should be sufficient to run configure without options: - - - - ./configure - - - -The configure script attempts to use use the C compiler specified by -the CC environment variable. If not set, GNU C will be -used if it is available. The CFLAGS environment variable -holds options to be passed to the C compiler. If you're using -Bourne-compatible shell you may pass something like this to use a -particular C compiler with optimization enabled: - - - - CC=/opt/ccs/bin/cc CFLAGS=-O ./configure - - - -To customize &yaz; the configure script also accepts a set of options. -The most important are: - - ---prefix path -Specifies installation prefix. This is -only needed if you run make install later to perform a -"system" installation. The prefix is /usr/local if not -specified. - - - ---enable-comp - &yaz; will be built using the ASN.1 compiler for &yaz; -(default). If you wish to use the old decoders (in sub directory asn) -use --disable-comp instead. - - ---enable-threads -&yaz; will be built using POSIX threads. -Specifically, _REENTRANT will be defined during -compilation. - - - - - - -When configured, build the software by typing: - - make - - - - - -The following files are generated by the make process: - -lib/libyaz.a - -The &yaz; programmers' library. - - -ztest/yaz-ztest -A test Z39.50 server. - - -client/yaz-client -A command mode Z39.50 client. - - -yaz-config -A Bourne-shell script that holds build -settings for &yaz;. - - -yaz-comp -The ASN.1 compiler for &yaz;. Requires the -Tcl Shell, tclsh, in current path to work. - - - - - - -If you wish to install &yaz; in system directories such as -/usr/local/bin, -/usr/local/lib you can type: - - - - make install - - - -You probably need to have root access in order to perform this. -You must specify the --prefix option for configure if -you wish to install &yaz; in other directories than the default -/usr/local/. - - - -If you wish to perform an un-installation of &yaz; use: - - - - make uninstall - - - -This will only work if you haven't reconfigured &yaz; (and therefore -changed installation prefix). Note that uninstall will not -remove directories created by make install, e.g. -/usr/local/include/yaz. - - - -WIN32 - - -&yaz; is shipped with "makefiles" for the NMAKE tool that comes -with Visual C++. - -Start an MS-DOS prompt and switch the sub directory WIN -where the file makefile is located. Customize the -installation by editing the makefile file (for example -by using notepad). - -The following summarises the most important settings in that file: - -WIN32 makefile settings - - - -Setting -Description - - - - - -NEW_Z3950 - If 1, the auto-generated decoder/encoders -for Z39.50 as written by the ASN.1 compiler will be used. If 0, the old -decoders for Z39.50 will be used. Note, when 1, the setting TCL should -point to the Tcl shell on your system. - - - - -DEBUG - If set to 1, the software is -compiled with debugging libraries. If set to 0, the software -is compiled with release (non-debugging) libraries. - - - - -TCL - Specifies the name of the Tcl shell (EXE-file). -You do not need setting this or installing Tcl unless you wish -to change or add ASN.1 for &yaz;. - - - - - -
- - - -When satisfied with the settings in the makefile type - - nmake - - - -The following files are generated upon successful compilation: - - -bin/yaz.dll - -the multi-threaded &yaz; DLL. - - -bin/yaz-ztest.exe - -A console Z39.50 client application. - - -bin/yaz-ztest.exe - -A console Z39.50 multi threaded server. - - - - - - - - + + Compilation and Installation + + + The latest version of the software will generally be found at + + + + http://ftp.indexdata.dk/pub/yaz/ + + + We have tried our best to keep the software portable, and on many + platforms, you should be able to compile everything with little or + no changes. + So far, the software has been ported to the following platforms with + little or no difficulties. + + + Unix systems + + HP/UX + SunOS/Solaris + DEC Unix + Linux + IBM AIX + Data General DG/UX (with some CFLAGS tinkering) + + SGI/IRIX + DDE Supermax + + Non-unix systems + + Apple Macintosh (using the Codewarrior programming + environment and the GUSI socket libraries) + MS Windows 95/98/NT/W2K (Win32) + IBM AS/400 + + + + + + If you move the software to other platforms, we'd be grateful if you'd + let us know about it. If you run into difficulties, we will try to help + if we can, and if you solve the problems, we would be happy to include + your fixes in the next release. So far, we have mostly avoided + #ifdefs for individual platforms, and we'd like to keep it that + way as far as it makes sense. + + + + We maintain a mailing-list for the purpose of announcing new releases and + bug-fixes, as well as general discussion. Subscribe by sending mail to + + yaz-request@indexdata.dk + . + General questions and problems can be directed at + + yaz-help@indexdata.dk + , or the address given at the top of this document. + + + UNIX + + + Note that if your system doesn't have a native ANSI C compiler, you may + have to acquire one separately. We recommend gcc. + + + For UNIX we use GNU configure to create Makefiles for &yaz;. + Generally it should be sufficient to run configure without options: + + + + ./configure + + + + The configure script attempts to use use the C compiler specified by + the CC environment variable. If not set, GNU C will be + used if it is available. The CFLAGS environment + variable holds options to be passed to the C compiler. If you're using + Bourne-compatible shell you may pass something like this to use a + particular C compiler with optimization enabled: + + + + CC=/opt/ccs/bin/cc CFLAGS=-O ./configure + + + + To customize &yaz; the configure script also accepts a set of options. + The most important are: + + + --prefix path + Specifies installation prefix. This is + only needed if you run make install later to + perform a "system" installation. The prefix is + /usr/local if not specified. + + + + --enable-comp + &yaz; will be built using the ASN.1 compiler for &yaz; + (default). If you wish to use the old decoders (in sub directory asn) + use --disable-comp instead. + + + --enable-threads + &yaz; will be built using POSIX threads. + Specifically, _REENTRANT will be defined during + compilation. + + + + + + + When configured, build the software by typing: + + make + + + + + + The following files are generated by the make process: + + lib/libyaz.a + + The &yaz; programmers' library. + + + ztest/yaz-ztest + A test Z39.50 server. + + + client/yaz-client + A command mode Z39.50 client. + + + yaz-config + A Bourne-shell script that holds build + settings for &yaz;. + + + yaz-comp + The ASN.1 compiler for &yaz;. Requires the + Tcl Shell, tclsh, in current path to work. + + + + + + + If you wish to install &yaz; in system directories such as + /usr/local/bin, + /usr/local/lib you can type: + + + + make install + + + + You probably need to have root access in order to perform this. + You must specify the --prefix option for configure if + you wish to install &yaz; in other directories than the default + /usr/local/. + + + + If you wish to perform an un-installation of &yaz; use: + + + + make uninstall + + + + This will only work if you haven't reconfigured &yaz; (and therefore + changed installation prefix). Note that uninstall will not + remove directories created by make install, e.g. + /usr/local/include/yaz. + + + + WIN32 + + + &yaz; is shipped with "makefiles" for the NMAKE tool that comes + with Visual C++. + + Start an MS-DOS prompt and switch the sub directory + WIN where the file makefile + is located. Customize the installation by editing the + makefile file (for example by using notepad). + + The following summarises the most important settings in that file: + + WIN32 makefile settings + + + + Setting + Description + + + + + + NEW_Z3950 + If 1, the auto-generated decoder/encoders + for Z39.50 as written by the ASN.1 compiler will be used. If 0, + the old decoders for Z39.50 will be used. Note, when 1, the + setting TCL should point to the Tcl shell on your system. + + + + + DEBUG + If set to 1, the software is + compiled with debugging libraries. If set to 0, the software + is compiled with release (non-debugging) libraries. + + + + + TCL + Specifies the name of the Tcl shell (EXE-file). + You do not need setting this or installing Tcl unless you wish + to change or add ASN.1 for &yaz;. + + + + + +
+ + + + When satisfied with the settings in the makefile type + + nmake + + + + The following files are generated upon successful compilation: + + + bin/yaz.dll + + the multi-threaded &yaz; DLL. + + + bin/yaz-ztest.exe + + A console Z39.50 client application. + + + bin/yaz-ztest.exe + + A console Z39.50 multi threaded server. + + + + + + + + + + \ No newline at end of file diff --git a/doc/introduction.xml b/doc/introduction.xml index 5cc082e..785bc82 100644 --- a/doc/introduction.xml +++ b/doc/introduction.xml @@ -1,93 +1,108 @@ - -Introduction + + Introduction + + + The &yaz; toolkit offers several different levels of access to the + Z39.50 and SR protocols. The level that you need to use depends on + your requirements, and the role (server or client) that you + want to implement. + + The basic level, which is independent of the role, consists of three + primary interfaces: + + + &asn;, which provides a C representation of the Z39.50/SR + protocol packages (PDUs). + + &odr;, which encodes and decodes the packages according + to the BER specification. + + &comstack;, which exchanges the encoded packages with + a peer process over a network. + + + + The &asn; module represents the ASN.1 definition of + the SR/Z39.50 protocol. It establishes a set of type and + structure definitions, with one structure for each of the top-level + PDUs, and one structure or type for each of the contained ASN.1 types. + For primitive types, or other types that are defined by the ASN.1 + standard itself (such as the EXTERNAL type), the C representation is + provided by the &odr; (Open Data Representation) subsystem. + + + &odr; is a basic mechanism for representing an + ASN.1 type in the C programming language, and for implementing BER + encoders and decoders for values of that type. The types defined in + the &asn; module generally have the prefix Z_, and + a suffix corresponding to the name of the type in the ASN.1 + specification of the protocol (generally Z39.50-1995). In the case of + base types (those originating in the ASN.1 standard itself), the prefix + Odr_ is sometimes seen. Either way, look for + the actual definition in either proto.h (for the types + from the protocol), odr.h (for the primitive ASN.1 + types, or odr_use.h (for the ASN.1 + useful types). The &asn; library also + provides functions (which are, in turn, defined using &odr; + primitives) for encoding and decoding data values. Their general form is - -The &yaz; toolkit offers several different levels of access to the -Z39.50 and SR protocols. The level that you need to use depends on -your requirements, and the role (server or client) that you -want to implement. - -The basic level, which is independent of the role, consists of three -primary interfaces: + + int z_xxx(ODR o, Z_xxx **p, int optional, const char *name); + + (note the lower-case "z" in the function name) + - -&asn;, which provides a C representation of the Z39.50/SR -protocol packages (PDUs). - -&odr;, which encodes and decodes the packages according -to the BER specification. - -&comstack;, which exchanges the encoded packages with -a peer process over a network. - - + + + If you are using the premade definitions of the &asn; module, and you + are not adding new protocol of your own, the only parts of &odr; that you + need to worry about are documented in section + Using ODR. + + -The &asn; module represents the ASN.1 definition of -the SR/Z39.50 protocol. It establishes a set of type and -structure definitions, with one structure for each of the top-level -PDUs, and one structure or type for each of the contained ASN.1 types. -For primitive types, or other types that are defined by the ASN.1 -standard itself (such as the EXTERNAL type), the C representation is provided -by the &odr; (Open Data Representation) subsystem. - - -&odr; is a basic mechanism for representing an -ASN.1 type in the C programming language, and for implementing BER -encoders and decoders for values of that type. The types defined in -the &asn; module generally have the prefix Z_, and a suffix -corresponding to the name of the type in the ASN.1 -specification of the protocol (generally Z39.50-1995). In the case of -base types (those originating in the ASN.1 standard itself), the prefix -Odr_ is sometimes seen. Either way, look for -the actual definition in either proto.h (for the types -from the protocol), odr.h (for the primitive ASN.1 -types, or odr_use.h (for the ASN.1 -useful types). The &asn; library also -provides functions (which are, in turn, defined using &odr; -primitives) for encoding and decoding data values. Their general form is + + When you have created a BER-encoded buffer, you can use the &comstack; + subsystem to transmit (or receive) data over the network. The &comstack; + module provides simple functions for establishing a connection + (passively or actively, depending on the role of your application), + and for exchanging BER-encoded PDUs over that connection. When you + create a connection endpoint, you need to specify what transport to + use (OSI or TCP/IP), and which protocol you want to use (SR or + Z39.50). For the remainer of the connection's lifetime, you don't have + to worry about the underlying transport protocol at all - the &comstack; + will ensure that the correct mechanism is used. + + + We call the combined interfaces to &odr;, &asn;, and &comstack; the service + level API. It's the API that most closely models the Z39.50/SR + service/protocol definition, and it provides unlimited access to all + fields and facilities of the protocol definitions. + + + The reason that the &yaz; service-level API is a conglomerate of the + APIs from three different submodules is twofold. First, we wanted to allow + the user a choice of different options for each major task. For instance, + if you don't like the protocol API provided by &odr;/&asn;, you + can use SNACC or BERUtils instead, and still have the benefits of the + transparent transport approach of the &comstack; module. Secondly, + we realise that you may have to fit the toolkit into an existing + event-processing structure, in a way that + is incompatible with the &comstack; interface or some other part of &yaz;. + + - -int z_xxx(ODR o, Z_xxx **p, int optional, const char *name); - -(note the lower-case "z" in the function name) - - - - -If you are using the premade definitions of the &asn; module, and you -are not adding new protocol of your own, the only parts of &odr; that you -need to worry about are documented in section -Using ODR. - - - - -When you have created a BER-encoded buffer, you can use the &comstack; -subsystem to transmit (or receive) data over the network. The &comstack; -module provides simple functions for establishing a connection -(passively or actively, depending on the role of your application), -and for exchanging BER-encoded PDUs over that connection. When you -create a connection endpoint, you need to specify what transport to -use (OSI or TCP/IP), and which protocol you want to use (SR or -Z39.50). For the remainer of the connection's lifetime, you don't have -to worry about the underlying transport protocol at all - the &comstack; -will ensure that the correct mechanism is used. - - -We call the combined interfaces to &odr;, &asn;, and &comstack; the service -level API. It's the API that most closely models the Z39.50/SR -service/protocol definition, and it provides unlimited access to all -fields and facilities of the protocol definitions. - - -The reason that the &yaz; service-level API is a conglomerate of the -APIs from three different submodules is twofold. First, we wanted to allow the -user a choice of different options for each major task. For instance, -if you don't like the protocol API provided by &odr;/&asn;, you -can use SNACC or BERUtils instead, and still have the benefits of the -transparent transport approach of the &comstack; module. Secondly, -we realise that you may have to fit the toolkit into an existing -event-processing structure, in a way that -is incompatible with the &comstack; interface or some other part of &yaz;. - - \ No newline at end of file + diff --git a/doc/license.xml b/doc/license.xml index 92ca04e..e07567d 100644 --- a/doc/license.xml +++ b/doc/license.xml @@ -1,103 +1,118 @@ - -License - -Index Data Copyright - - -Copyright © 1995-2001 Index Data. - - - -Permission to use, copy, modify, distribute, and sell this software and -its documentation, in whole or in part, for any purpose, is hereby granted, -provided that: - - - -1. This copyright and permission notice appear in all copies of the -software and its documentation. Notices of copyright or attribution -which appear at the beginning of any file must remain unchanged. - - - -2. The names of Index Data or the individual authors may not be used to -endorse or promote products derived from this software without specific -prior written permission. - - - -THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND, -EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY -WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. -IN NO EVENT SHALL INDEX DATA BE LIABLE FOR ANY SPECIAL, INCIDENTAL, -INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES -WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR -NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF -LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -OF THIS SOFTWARE. - - -Additional Copyright Statements - - -The optional CCL query language interpreter is covered by the following -license: - - - -Copyright © 1995, the EUROPAGATE consortium (see below). - - - -The EUROPAGATE consortium members are: - University College Dublin - Danmarks Teknologiske Videnscenter - An Chomhairle Leabharlanna - Consejo Superior de Investigaciones Cientificas - - - -Permission to use, copy, modify, distribute, and sell this software and -its documentation, in whole or in part, for any purpose, is hereby granted, -provided that: - - - -1. This copyright and permission notice appear in all copies of the -software and its documentation. Notices of copyright or attribution -which appear at the beginning of any file must remain unchanged. - - - -2. The names of EUROPAGATE or the project partners may not be used to -endorse or promote products derived from this software without specific -prior written permission. - - - -3. Users of this software (implementors and gateway operators) agree to -inform the EUROPAGATE consortium of their use of the software. This -information will be used to evaluate the EUROPAGATE project and the -software, and to plan further developments. The consortium may use -the information in later publications. - - - -4. Users of this software agree to make their best efforts, when -documenting their use of the software, to acknowledge the EUROPAGATE -consortium, and the role played by the software in their work. - - - -THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND, -EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY -WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. -IN NO EVENT SHALL THE EUROPAGATE CONSORTIUM OR ITS MEMBERS BE LIABLE -FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF -ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA -OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND -ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE -USE OR PERFORMANCE OF THIS SOFTWARE. - - - \ No newline at end of file + + License + + Index Data Copyright + + + Copyright © 1995-2001 Index Data. + + + + Permission to use, copy, modify, distribute, and sell this software and + its documentation, in whole or in part, for any purpose, is hereby granted, + provided that: + + + + 1. This copyright and permission notice appear in all copies of the + software and its documentation. Notices of copyright or attribution + which appear at the beginning of any file must remain unchanged. + + + + 2. The names of Index Data or the individual authors may not be used to + endorse or promote products derived from this software without specific + prior written permission. + + + + THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND, + EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY + WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + IN NO EVENT SHALL INDEX DATA BE LIABLE FOR ANY SPECIAL, INCIDENTAL, + INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR + NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF + LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE + OF THIS SOFTWARE. + + + Additional Copyright Statements + + + The optional CCL query language interpreter is covered by the following + license: + + + + Copyright © 1995, the EUROPAGATE consortium (see below). + + + + The EUROPAGATE consortium members are: + University College Dublin + Danmarks Teknologiske Videnscenter + An Chomhairle Leabharlanna + Consejo Superior de Investigaciones Cientificas + + + + Permission to use, copy, modify, distribute, and sell this software and + its documentation, in whole or in part, for any purpose, is hereby granted, + provided that: + + + + 1. This copyright and permission notice appear in all copies of the + software and its documentation. Notices of copyright or attribution + which appear at the beginning of any file must remain unchanged. + + + + 2. The names of EUROPAGATE or the project partners may not be used to + endorse or promote products derived from this software without specific + prior written permission. + + + + 3. Users of this software (implementors and gateway operators) agree to + inform the EUROPAGATE consortium of their use of the software. This + information will be used to evaluate the EUROPAGATE project and the + software, and to plan further developments. The consortium may use + the information in later publications. + + + + 4. Users of this software agree to make their best efforts, when + documenting their use of the software, to acknowledge the EUROPAGATE + consortium, and the role played by the software in their work. + + + + THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND, + EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY + WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + IN NO EVENT SHALL THE EUROPAGATE CONSORTIUM OR ITS MEMBERS BE LIABLE + FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF + ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA + OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND + ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE + USE OR PERFORMANCE OF THIS SOFTWARE. + + + + + diff --git a/doc/odr.xml b/doc/odr.xml index 87af9f8..de0b3ca 100644 --- a/doc/odr.xml +++ b/doc/odr.xml @@ -1,329 +1,333 @@ - -The ODR Module - -Introduction - - -&odr; is the BER-encoding/decoding subsystem of &yaz;. Care as been taken -to isolate &odr; from the rest of the package - specifically from the -transport interface. &odr; may be used in any context where basic -ASN.1/BER representations are used. - - - -If you are only interested in writing a Z39.50 implementation based on -the PDUs that are already provided with &yaz;, you only need to concern -yourself with the section on managing ODR streams (section -Using ODR). Only if you need to -implement ASN.1 beyond that which has been provided, should you -worry about the second half of the documentation -(section Programming with ODR). -If you use one of the higher-level interfaces, you can skip this -section entirely. - - - -This is important, so we'll repeat it for emphasis: You do not -need to read section Programming with ODR to -implement Z39.50 with &yaz;. - - - -If you need a part of the protocol that isn't already in &yaz;, you -should contact the authors before going to work on it yourself: We -might already be working on it. Conversely, if you implement a useful -part of the protocol before us, we'd be happy to include it in a -future release. - - - -Using ODR - -ODR Streams - - -Conceptually, the ODR stream is the source of encoded data in the -decoding mode; when encoding, it is the receptacle for the encoded -data. Before you can use an ODR stream it must be allocated. This is -done with the function - - - - ODR odr_createmem(int direction); - - - -The odr_createmem() function takes as argument one -of three manifest constants: ODR_ENCODE, -ODR_DECODE, or ODR_PRINT. -An &odr; stream can be in only one mode - it is not possible to change -its mode once it's selected. Typically, your program will allocate -at least two ODR streams - one for decoding, and one for encoding. - - - -When you're done with the stream, you can use - - - - void odr_destroy(ODR o); - - - -to release the resources allocated for the stream. - - - -Memory Management - - -Two forms of memory management take place in the &odr; system. The first -one, which has to do with allocating little bits of memory (sometimes -quite large bits of memory, actually) when a protocol package is -decoded, and turned into a complex of interlinked structures. This -section deals with this system, and how you can use it for your own -purposes. The next section deals with the memory management which is -required when encoding data - to make sure that a large enough buffer is -available to hold the fully encoded PDU. - - - -The &odr; module has its own memory management system, which is -used whenever memory is required. Specifically, it is used to allocate -space for data when decoding incoming PDUs. You can use the memory -system for your own purposes, by using the function - - - -void *odr_malloc(ODR o, int size); - - - -You can't use the normal free(2) routine to free -memory allocated by this function, and &odr; doesn't provide a parallel -function. Instead, you can call - - - - void odr_reset(ODR o, int size); - - - -when you are done with the -memory: Everything allocated since the last call to -odr_reset() is released. -The odr_reset() call is also required to clear -up an error condition on a stream. - - - -The function - - - - int odr_total(ODR o); - - - -returns the number of bytes allocated on the stream since the last call to -odr_reset(). - - - -The memory subsystem of &odr; is fairly efficient at allocating and -releasing little bits of memory. Rather than managing the individual, -small bits of space, the system maintains a freelist of larger chunks -of memory, which are handed out in small bits. This scheme is -generally known as a nibble memory system. -It is very useful for maintaing short-lived constructions such -as protocol PDUs. - - - -If you want to retain a bit of memory beyond the next call to -odr_reset(), you can use the function - - - - ODR_MEM odr_extract_mem(ODR o); - - - -This function will give you control of the memory recently allocated -on the ODR stream. The memory will live (past calls to -odr_reset()), until you call the function - - - - void odr_release_mem(ODR_MEM p); - - - -The opaque ODR_MEM handle has no other purpose than -referencing the memory block for you until you want to release it. - - - -You can use odr_extract_mem() repeatedly between -allocating data, to retain individual control of separate chunks of data. - - - -Encoding and Decoding Data - - -When encoding data, the ODR stream will write the encoded octet string -in an internal buffer. To retrieve the data, use the function - - - - char *odr_getbuf(ODR o, int *len, int *size); - - - -The integer pointed to by len is set to the length of the encoded -data, and a pointer to that data is returned. *size -is set to the size of the buffer (unless size is null, -signalling that you are not interested in the size). The next call to -a primitive function using the same &odr; stream will overwrite the -data, unless a different buffer has been supplied using the call - - - - void odr_setbuf(ODR o, char *buf, int len, int can_grow); - - - -which sets the encoding (or decoding) buffer used by o to -buf, using the length len. -Before a call to an encoding function, you can use -odr_setbuf() to provide the stream with an encoding -buffer of sufficient size (length). The can_grow -parameter tells the encoding &odr; stream whether it is allowed to use -realloc(2) to increase the size of the buffer when -necessary. The default condition of a new encoding stream is equivalent -to the results of calling - - - -odr_setbuf(stream, 0, 0, 1); - - - -In this case, the stream will allocate and reallocate memory as -necessary. The stream reallocates memory by repeatedly doubling the -size of the buffer - the result is that the buffer will typically -reach its maximum, working size with only a small number of reallocation -operations. The memory is freed by the stream when the latter is destroyed, -unless it was assigned by the user with the can_grow -parameter set to zero (in this case, you are expected to retain -control of the memory yourself). - - - -To assume full control of an encoded buffer, you must first call -odr_getbuf() to fetch the buffer and its length. -Next, you should call odr_setbuf() to provide a -different buffer (or a null pointer) to the stream. In the simplest -case, you will reuse the same buffer over and over again, and you -will just need to call odr_getbuf() after each -encoding operation to get the length and address of the buffer. -Note that the stream may reallocate the buffer during an encoding -operation, so it is necessary to retrieve the correct address after -each encoding operation. - - - -It is important to realise that the ODR stream will not release this -memory when you call odr_reset(): It will -merely update its internal pointers to prepare for the encoding of a -new data value. -When the stream is released by the odr_destroy() -function, the memory given to it by odr_setbuf will -be released only if the can_grow -parameter to odr_setbuf() was nonzero. The -can_grow parameter, in other words, is a way of -signalling who is to own the buffer, you or the ODR stream. If you never call -odr_setbuf() on your encoding stream, which is -typically the case, the buffer allocated by the stream will belong to -the stream by default. - - - -When you wish to decode data, you should first call -odr_setbuf(), to tell the decoding stream -where to find the encoded data, and how long the buffer is -(the can_grow parameter is ignored by a decoding -stream). After this, you can call the function corresponding to the -data you wish to decode (eg, odr_integer() odr -z_APDU()). - - - -Examples of encoding/decoding functions: - - - - int odr_integer(ODR o, int **p, int optional, const char *name); - - int z_APDU(ODR o, Z_APDU **p, int optional, const char *name); - - - -If the data is absent (or doesn't match the tag corresponding to the type), -the return value will be either 0 or 1 depending on the -optional flag. If optional -is 0 and the data is absent, an error flag will be raised in the -stream, and you'll need to call odr_reset() before -you can use the stream again. If optional is -nonzero, the pointer pointed to/ by p -will be set to the null value, and the function will return 1. -The name argument is used to pretty-print the -tag in question. It may be set to NULL if -pretty-printing is not desired. - - - -If the data value is found where it's expected, the pointer -pointed to by the p argument -will be set to point to the decoded type. -The space for the type will be allocated and owned by the &odr; stream, and -it will live until you call odr_reset() on the -stream. You cannot use free(2) to release the memory. -You can decode several data elements (by repeated calls to -odr_setbuf() and your decoding function), and -new memory will be allocated each time. When you do call -odr_reset(), everything decoded since the -last call to odr_reset() will be released. - - - -The use of the double indirection can be a little confusing at first -(its purpose will become clear later on, hopefully), -so an example is in order. We'll encode an integer value, and -immediately decode it again using a different stream. A useless, but -informative operation. - - - + + The ODR Module + + Introduction + + + &odr; is the BER-encoding/decoding subsystem of &yaz;. Care as been taken + to isolate &odr; from the rest of the package - specifically from the + transport interface. &odr; may be used in any context where basic + ASN.1/BER representations are used. + + + + If you are only interested in writing a Z39.50 implementation based on + the PDUs that are already provided with &yaz;, you only need to concern + yourself with the section on managing ODR streams (section + Using ODR). Only if you need to + implement ASN.1 beyond that which has been provided, should you + worry about the second half of the documentation + (section Programming with ODR). + If you use one of the higher-level interfaces, you can skip this + section entirely. + + + + This is important, so we'll repeat it for emphasis: You do not + need to read section Programming with ODR to + implement Z39.50 with &yaz;. + + + + If you need a part of the protocol that isn't already in &yaz;, you + should contact the authors before going to work on it yourself: We + might already be working on it. Conversely, if you implement a useful + part of the protocol before us, we'd be happy to include it in a + future release. + + + + Using ODR + + ODR Streams + + + Conceptually, the ODR stream is the source of encoded data in the + decoding mode; when encoding, it is the receptacle for the encoded + data. Before you can use an ODR stream it must be allocated. This is + done with the function + + + + ODR odr_createmem(int direction); + + + + The odr_createmem() function takes as argument one + of three manifest constants: ODR_ENCODE, + ODR_DECODE, or ODR_PRINT. + An &odr; stream can be in only one mode - it is not possible to change + its mode once it's selected. Typically, your program will allocate + at least two ODR streams - one for decoding, and one for encoding. + + + + When you're done with the stream, you can use + + + + void odr_destroy(ODR o); + + + + to release the resources allocated for the stream. + + + + Memory Management + + + Two forms of memory management take place in the &odr; system. The first + one, which has to do with allocating little bits of memory (sometimes + quite large bits of memory, actually) when a protocol package is + decoded, and turned into a complex of interlinked structures. This + section deals with this system, and how you can use it for your own + purposes. The next section deals with the memory management which is + required when encoding data - to make sure that a large enough buffer is + available to hold the fully encoded PDU. + + + + The &odr; module has its own memory management system, which is + used whenever memory is required. Specifically, it is used to allocate + space for data when decoding incoming PDUs. You can use the memory + system for your own purposes, by using the function + + + + void *odr_malloc(ODR o, int size); + + + + You can't use the normal free(2) routine to free + memory allocated by this function, and &odr; doesn't provide a parallel + function. Instead, you can call + + + + void odr_reset(ODR o, int size); + + + + when you are done with the + memory: Everything allocated since the last call to + odr_reset() is released. + The odr_reset() call is also required to clear + up an error condition on a stream. + + + + The function + + + + int odr_total(ODR o); + + + + returns the number of bytes allocated on the stream since the last call to + odr_reset(). + + + + The memory subsystem of &odr; is fairly efficient at allocating and + releasing little bits of memory. Rather than managing the individual, + small bits of space, the system maintains a freelist of larger chunks + of memory, which are handed out in small bits. This scheme is + generally known as a nibble memory system. + It is very useful for maintaing short-lived constructions such + as protocol PDUs. + + + + If you want to retain a bit of memory beyond the next call to + odr_reset(), you can use the function + + + + ODR_MEM odr_extract_mem(ODR o); + + + + This function will give you control of the memory recently allocated + on the ODR stream. The memory will live (past calls to + odr_reset()), until you call the function + + + + void odr_release_mem(ODR_MEM p); + + + + The opaque ODR_MEM handle has no other purpose than + referencing the memory block for you until you want to release it. + + + + You can use odr_extract_mem() repeatedly between + allocating data, to retain individual control of separate chunks of data. + + + + Encoding and Decoding Data + + + When encoding data, the ODR stream will write the encoded octet string + in an internal buffer. To retrieve the data, use the function + + + + char *odr_getbuf(ODR o, int *len, int *size); + + + + The integer pointed to by len is set to the length of the encoded + data, and a pointer to that data is returned. *size + is set to the size of the buffer (unless size is null, + signalling that you are not interested in the size). The next call to + a primitive function using the same &odr; stream will overwrite the + data, unless a different buffer has been supplied using the call + + + + void odr_setbuf(ODR o, char *buf, int len, int can_grow); + + + + which sets the encoding (or decoding) buffer used by + o to buf, using the length + len. + Before a call to an encoding function, you can use + odr_setbuf() to provide the stream with an encoding + buffer of sufficient size (length). The can_grow + parameter tells the encoding &odr; stream whether it is allowed to use + realloc(2) to increase the size of the buffer when + necessary. The default condition of a new encoding stream is equivalent + to the results of calling + + + + odr_setbuf(stream, 0, 0, 1); + + + + In this case, the stream will allocate and reallocate memory as + necessary. The stream reallocates memory by repeatedly doubling the + size of the buffer - the result is that the buffer will typically + reach its maximum, working size with only a small number of reallocation + operations. The memory is freed by the stream when the latter is destroyed, + unless it was assigned by the user with the can_grow + parameter set to zero (in this case, you are expected to retain + control of the memory yourself). + + + + To assume full control of an encoded buffer, you must first call + odr_getbuf() to fetch the buffer and its length. + Next, you should call odr_setbuf() to provide a + different buffer (or a null pointer) to the stream. In the simplest + case, you will reuse the same buffer over and over again, and you + will just need to call odr_getbuf() after each + encoding operation to get the length and address of the buffer. + Note that the stream may reallocate the buffer during an encoding + operation, so it is necessary to retrieve the correct address after + each encoding operation. + + + + It is important to realise that the ODR stream will not release this + memory when you call odr_reset(): It will + merely update its internal pointers to prepare for the encoding of a + new data value. + When the stream is released by the odr_destroy() + function, the memory given to it by odr_setbuf will + be released only if the can_grow + parameter to odr_setbuf() was nonzero. The + can_grow parameter, in other words, is a way of + signalling who is to own the buffer, you or the ODR stream. If you never call + odr_setbuf() on your encoding stream, which is + typically the case, the buffer allocated by the stream will belong to + the stream by default. + + + + When you wish to decode data, you should first call + odr_setbuf(), to tell the decoding stream + where to find the encoded data, and how long the buffer is + (the can_grow parameter is ignored by a decoding + stream). After this, you can call the function corresponding to the + data you wish to decode (eg, odr_integer() odr + z_APDU()). + + + + Examples of encoding/decoding functions: + + + + int odr_integer(ODR o, int **p, int optional, const char *name); + + int z_APDU(ODR o, Z_APDU **p, int optional, const char *name); + + + + If the data is absent (or doesn't match the tag corresponding to + the type), the return value will be either 0 or 1 depending on the + optional flag. If optional + is 0 and the data is absent, an error flag will be raised in the + stream, and you'll need to call odr_reset() before + you can use the stream again. If optional is + nonzero, the pointer pointed to/ by + p will be set to the null value, and the function + will return 1. + The name argument is used to pretty-print the + tag in question. It may be set to NULL if + pretty-printing is not desired. + + + + If the data value is found where it's expected, the pointer + pointed to by the p argument + will be set to point to the decoded type. + The space for the type will be allocated and owned by the &odr; + stream, and it will live until you call + odr_reset() on the stream. You cannot use + free(2) to release the memory. + You can decode several data elements (by repeated calls to + odr_setbuf() and your decoding function), and + new memory will be allocated each time. When you do call + odr_reset(), everything decoded since the + last call to odr_reset() will be released. + + + + The use of the double indirection can be a little confusing at first + (its purpose will become clear later on, hopefully), + so an example is in order. We'll encode an integer value, and + immediately decode it again using a different stream. A useless, but + informative operation. + + + + void do_nothing_useful(int value) { ODR encode, decode; int *valp, *resvalp; char *bufferp; int len; - + /* allocate streams */ if (!(encode = odr_createmem(ODR_ENCODE))) - return; + return; if (!(decode = odr_createmem(ODR_DECODE))) - return; + return; valp = &value; if (odr_integer(encode, &valp, 0, 0) == 0) { - printf("encoding went bad\n"); - return; + printf("encoding went bad\n"); + return; } bufferp = odr_getbuf(encode, &len); printf("length of encoded data is %d\n", len); @@ -332,8 +336,8 @@ void do_nothing_useful(int value) odr_setbuf(decode, bufferp, len); if (odr_integer(decode, &resvalp, 0, 0) == 0) { - printf("decoding went bad\n"); - return; + printf("decoding went bad\n"); + return; } printf("the value is %d\n", *resvalp); @@ -341,809 +345,817 @@ void do_nothing_useful(int value) odr_destroy(encode); odr_destroy(decode); } - - - -This looks like a lot of work, offhand. In practice, the &odr; streams -will typically be allocated once, in the beginning of your program (or at the -beginning of a new network session), and the encoding and decoding -will only take place in a few, isolated places in your program, so the -overhead is quite manageable. - - - - -Diagnostics - - -The encoding/decoding functions all return 0 when an error occurs. -Until you call odr_reset(), you cannot use the -stream again, and any function called will immediately return 0. - - - -To provide information to the programmer or administrator, the function - - - - void odr_perror(ODR o, char *message); - - - -is provided, which prints the message argument to -stderr along with an error message from the stream. - + + + + This looks like a lot of work, offhand. In practice, the &odr; streams + will typically be allocated once, in the beginning of your program + (or at the beginning of a new network session), and the encoding + and decoding will only take place in a few, isolated places in your + program, so the overhead is quite manageable. + + + + + Diagnostics + + + The encoding/decoding functions all return 0 when an error occurs. + Until you call odr_reset(), you cannot use the + stream again, and any function called will immediately return 0. + + + + To provide information to the programmer or administrator, the function + + + + void odr_perror(ODR o, char *message); + + + + is provided, which prints the message argument to + stderr along with an error message from the stream. + - -You can also use the function - + + You can also use the function + - - int odr_geterror(ODR o); - + + int odr_geterror(ODR o); + - -to get the current error number from the screen. The number will be -one of these constants: - + + to get the current error number from the screen. The number will be + one of these constants: + -ODR Error codes - - - -code -Description - - - - -OMEMORYMemory allocation failed. - - - -OSYSERRA system- or library call has failed. -The standard diagnostic variable errno should be -examined to determine the actual error. - - - -OSPACENo more space for encoding. -This will only occur when the user has explicitly provided a -buffer for an encoding stream without allowing the system to -allocate more space. - - - -OREQUIREDThis is a common protocol error; A -required data element was missing during encoding or decoding. - - - -OUNEXPECTEDAn unexpected data element was -found during decoding. - - -OOTHEROther error. This is typically an -indication of misuse of the &odr; system by the programmer, and also -that the diagnostic system isn't as good as it should be, yet. - - - -
- - -The character string array - - - - char *odr_errlist[] - - - -can be indexed by the error code to obtain a human-readable -representation of the problem. - - - -Summary and Synopsis - - -#include <odr.h> - -ODR odr_createmem(int direction); - -void odr_destroy(ODR o); - -void odr_reset(ODR o); - -char *odr_getbuf(ODR o, int *len); - -void odr_setbuf(ODR o, char *buf, int len); - -void *odr_malloc(ODR o, int size); - -ODR_MEM odr_extract_mem(ODR o); - -void odr_release_mem(ODR_MEM r); - -int odr_geterror(ODR o); - -void odr_perror(char *message); - -extern char *odr_errlist[]; - - - - - -Programming with ODR - - -The API of &odr; is designed to reflect the structure of ASN.1, rather -than BER itself. Future releases may be able to represent data in -other external forms. - - - -The interface is based loosely on that of the Sun Microsystems XDR routines. -Specifically, each function which corresponds to an ASN.1 primitive -type has a dual function. Depending on the settings of the ODR -stream which is supplied as a parameter, the function may be used -either to encode or decode data. The functions that can be built -using these primitive functions, to represent more complex datatypes, share -this quality. The result is that you only have to enter the definition -for a type once - and you have the functionality of encoding, decoding -(and pretty-printing) all in one unit. The resulting C source code is -quite compact, and is a pretty straightforward representation of the -source ASN.1 specification. Although no ASN.1 compiler is supplied -with &odr; at this time, it shouldn't be too difficult to write one, or -perhaps even to adapt an existing compiler to output &odr; routines -(not surprisingly, writing encoders/decoders using &odr; turns out -to be boring work). - - - -In many cases, the model of the XDR functions works quite well in this -role. -In others, it is less elegant. Most of the hassle comes from the optional -SEQUENCE memebers which don't exist in XDR. - - -The Primitive ASN.1 Types - - -ASN.1 defines a number of primitive types (many of which correspond -roughly to primitive types in structured programming languages, such as C). - - -INTEGER - - -The &odr; function for encoding or decoding (or printing) the ASN.1 -INTEGER type looks like this: - - - - int odr_integer(ODR o, int **p, int optional, const char *name); - - - -(we don't allow values that can't be contained in a C integer.) - - - -This form is typical of the primitive &odr; functions. They are named -after the type of data that they encode or decode. They take an &odr; -stream, an indirect reference to the type in question, and an -optional flag (corresponding to the OPTIONAL keyword -of ASN.1) as parameters. They all return an integer value of either one -or zero. -When you use the primitive functions to construct encoders for complex -types of your own, you should follow this model as well. This -ensures that your new types can be reused as elements in yet more -complex types. - - - -The o parameter should obviously refer to a properly -initialized &odr; stream of the right type (encoding/decoding/printing) -for the operation that you wish to perform. - - - -When encoding or printing, the function first looks at -* p. If * p (the pointer pointed -to by p) is a null pointer, this is taken to mean that -the data element is absent. If the optional parameter -is nonzero, the function will return one (signifying success) without -any further processing. If the optional is zero, an -internal error flag is set in the &odr; stream, and the function will -return 0. No further operations can be carried out on the stream without -a call to the function odr_reset(). - - - -If *p is not a null pointer, it is expected to -point to an instance of the data type. The data will be subjected to -the encoding rules, and the result will be placed in the buffer held -by the &odr; stream. - - - -The other ASN.1 primitives have similar functions that operate in -similar manners: - - -BOOLEAN - - - int odr_bool(ODR o, bool_t **p, int optional, const char *name); - - - -REAL - - -Not defined. - - - -NULL - - - int odr_null(ODR o, bool_t **p, int optional, const char *name); - - - -In this case, the value of **p is not important. If *p -is different from the null pointer, the null value is present, otherwise -it's absent. - - - -OCTET STRING - - - typedef struct odr_oct - { - unsigned char *buf; - int len; - int size; - } Odr_oct; - - int odr_octetstring(ODR o, Odr_oct **p, int optional, const char *name); - - - -The buf field should point to the character array -that holds the octetstring. The len field holds the -actual length, while the size field gives the size -of the allocated array (not of interest to you, in most cases). -The character array need not be null terminated. - - - -To make things a little easier, an alternative is given for string -types that are not expected to contain embedded NULL characters (eg. -VisibleString): - - - - int odr_cstring(ODR o, char **p, int optional, const char *name); - - - -Which encoded or decodes between OCTETSTRING representations and -null-terminates C strings. - - - -Functions are provided for the derived string types, eg: - - - - int odr_visiblestring(ODR o, char **p, int optional, const char *name); - - - -BIT STRING - - - int odr_bitstring(ODR o, Odr_bitmask **p, int optional, const char *name); - - - -The opaque type Odr_bitmask is only suitable for -holding relatively brief bit strings, eg. for options fields, etc. -The constant ODR_BITMASK_SIZE multiplied by 8 -gives the maximum possible number of bits. - - - -A set of macros are provided for manipulating the -Odr_bitmask type: - - - - void ODR_MASK_ZERO(Odr_bitmask *b); - - void ODR_MASK_SET(Odr_bitmask *b, int bitno); - - void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno); - - int ODR_MASK_GET(Odr_bitmask *b, int bitno); - - - -The functions are modelled after the manipulation functions that -accompany the fd_set type used by the -select(2) call. -ODR_MASK_ZERO should always be called first on a -new bitmask, to initialize the bits to zero. - - - -OBJECT IDENTIFIER - - + ODR Error codes + + + + code + Description + + + + + OMEMORYMemory allocation failed. + + + + OSYSERRA system- or library call has failed. + The standard diagnostic variable errno should be + examined to determine the actual error. + + + + OSPACENo more space for encoding. + This will only occur when the user has explicitly provided a + buffer for an encoding stream without allowing the system to + allocate more space. + + + + OREQUIREDThis is a common protocol error; A + required data element was missing during encoding or decoding. + + + + OUNEXPECTEDAn unexpected data element was + found during decoding. + + + OOTHEROther error. This is typically an + indication of misuse of the &odr; system by the programmer, and also + that the diagnostic system isn't as good as it should be, yet. + + + +
+ + + The character string array + + + + char *odr_errlist[] + + + + can be indexed by the error code to obtain a human-readable + representation of the problem. + + + + Summary and Synopsis + + + #include <odr.h> + + ODR odr_createmem(int direction); + + void odr_destroy(ODR o); + + void odr_reset(ODR o); + + char *odr_getbuf(ODR o, int *len); + + void odr_setbuf(ODR o, char *buf, int len); + + void *odr_malloc(ODR o, int size); + + ODR_MEM odr_extract_mem(ODR o); + + void odr_release_mem(ODR_MEM r); + + int odr_geterror(ODR o); + + void odr_perror(char *message); + + extern char *odr_errlist[]; + + + + + + Programming with ODR + + + The API of &odr; is designed to reflect the structure of ASN.1, rather + than BER itself. Future releases may be able to represent data in + other external forms. + + + + The interface is based loosely on that of the Sun Microsystems XDR routines. + Specifically, each function which corresponds to an ASN.1 primitive + type has a dual function. Depending on the settings of the ODR + stream which is supplied as a parameter, the function may be used + either to encode or decode data. The functions that can be built + using these primitive functions, to represent more complex datatypes, share + this quality. The result is that you only have to enter the definition + for a type once - and you have the functionality of encoding, decoding + (and pretty-printing) all in one unit. The resulting C source code is + quite compact, and is a pretty straightforward representation of the + source ASN.1 specification. Although no ASN.1 compiler is supplied + with &odr; at this time, it shouldn't be too difficult to write one, or + perhaps even to adapt an existing compiler to output &odr; routines + (not surprisingly, writing encoders/decoders using &odr; turns out + to be boring work). + + + + In many cases, the model of the XDR functions works quite well in this + role. + In others, it is less elegant. Most of the hassle comes from the optional + SEQUENCE memebers which don't exist in XDR. + + + The Primitive ASN.1 Types + + + ASN.1 defines a number of primitive types (many of which correspond + roughly to primitive types in structured programming languages, such as C). + + + INTEGER + + + The &odr; function for encoding or decoding (or printing) the ASN.1 + INTEGER type looks like this: + + + +int odr_integer(ODR o, int **p, int optional, const char *name); + + + + (we don't allow values that can't be contained in a C integer.) + + + + This form is typical of the primitive &odr; functions. They are named + after the type of data that they encode or decode. They take an &odr; + stream, an indirect reference to the type in question, and an + optional flag (corresponding to the OPTIONAL keyword + of ASN.1) as parameters. They all return an integer value of either one + or zero. + When you use the primitive functions to construct encoders for complex + types of your own, you should follow this model as well. This + ensures that your new types can be reused as elements in yet more + complex types. + + + + The o parameter should obviously refer to a properly + initialized &odr; stream of the right type (encoding/decoding/printing) + for the operation that you wish to perform. + + + + When encoding or printing, the function first looks at + * p. If * p (the pointer pointed + to by p) is a null pointer, this is taken to mean that + the data element is absent. If the optional parameter + is nonzero, the function will return one (signifying success) without + any further processing. If the optional is zero, an + internal error flag is set in the &odr; stream, and the function will + return 0. No further operations can be carried out on the stream without + a call to the function odr_reset(). + + + + If *p is not a null pointer, it is expected to + point to an instance of the data type. The data will be subjected to + the encoding rules, and the result will be placed in the buffer held + by the &odr; stream. + + + + The other ASN.1 primitives have similar functions that operate in + similar manners: + + + BOOLEAN + + +int odr_bool(ODR o, bool_t **p, int optional, const char *name); + + + + REAL + + + Not defined. + + + + NULL + + +int odr_null(ODR o, bool_t **p, int optional, const char *name); + + + + In this case, the value of **p is not important. If *p + is different from the null pointer, the null value is present, otherwise + it's absent. + + + + OCTET STRING + + +typedef struct odr_oct +{ + unsigned char *buf; + int len; + int size; +} Odr_oct; + +int odr_octetstring(ODR o, Odr_oct **p, int optional, + const char *name); + + + + The buf field should point to the character array + that holds the octetstring. The len field holds the + actual length, while the size field gives the size + of the allocated array (not of interest to you, in most cases). + The character array need not be null terminated. + + + + To make things a little easier, an alternative is given for string + types that are not expected to contain embedded NULL characters (eg. + VisibleString): + + + + int odr_cstring(ODR o, char **p, int optional, const char *name); + + + + Which encoded or decodes between OCTETSTRING representations and + null-terminates C strings. + + + + Functions are provided for the derived string types, eg: + + + +int odr_visiblestring(ODR o, char **p, int optional, + const char *name); + + + + BIT STRING + + +int odr_bitstring(ODR o, Odr_bitmask **p, int optional, + const char *name); + + + + The opaque type Odr_bitmask is only suitable for + holding relatively brief bit strings, eg. for options fields, etc. + The constant ODR_BITMASK_SIZE multiplied by 8 + gives the maximum possible number of bits. + + + + A set of macros are provided for manipulating the + Odr_bitmask type: + + + +void ODR_MASK_ZERO(Odr_bitmask *b); + +void ODR_MASK_SET(Odr_bitmask *b, int bitno); + +void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno); + +int ODR_MASK_GET(Odr_bitmask *b, int bitno); + + + + The functions are modelled after the manipulation functions that + accompany the fd_set type used by the + select(2) call. + ODR_MASK_ZERO should always be called first on a + new bitmask, to initialize the bits to zero. + + + + OBJECT IDENTIFIER + + int odr_oid(ODR o, Odr_oid **p, int optional, const char *name); - - - -The C OID represenation is simply an array of integers, terminated by -the value -1 (the Odr_oid type is synonymous with -the int type). -We suggest that you use the OID database module (see section -Object Identifiers) to handle object identifiers -in your application. - - - - -Tagging Primitive Types - - -The simplest way of tagging a type is to use the -odr_implicit_tag() or -odr_explicit_tag() macros: - - - - int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag, int - optional, const char *name); - - int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag, - int optional, const char *name); - - - -To create a type derived from the integer type by implicit tagging, you -might write: - - - - MyInt ::= [210] IMPLICIT INTEGER - - - -In the &odr; system, this would be written like: - - - + + + + The C OID represenation is simply an array of integers, terminated by + the value -1 (the Odr_oid type is synonymous with + the int type). + We suggest that you use the OID database module (see section + Object Identifiers) to handle object identifiers + in your application. + + + + + Tagging Primitive Types + + + The simplest way of tagging a type is to use the + odr_implicit_tag() or + odr_explicit_tag() macros: + + + +int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag, + int optional, const char *name); + +int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag, + int optional, const char *name); + + + + To create a type derived from the integer type by implicit tagging, you + might write: + + + + MyInt ::= [210] IMPLICIT INTEGER + + + + In the &odr; system, this would be written like: + + + int myInt(ODR o, int **p, int optional, const char *name) { return odr_implicit_tag(o, odr_integer, p, - ODR_CONTEXT, 210, optional, name); + ODR_CONTEXT, 210, optional, name); } - - - -The function myInt() can then be used like any of -the primitive functions provided by &odr;. Note that the behavior of -odr_explicit() -and odr_implicit() macros -act exactly the same as the functions they are applied to - they -respond to error conditions, etc, in the same manner - they -simply have three extra parameters. The class parameter may -take one of the values: ODR_CONTEXT, -ODR_PRIVATE, ODR_UNIVERSAL, or -/ODR_APPLICATION. - - - -Constructed Types - - -Constructed types are created by combining primitive types. The -&odr; system only implements the SEQUENCE and SEQUENCE OF constructions -(although adding the rest of the container types should be simple -enough, if the need arises). - - - -For implementing SEQUENCEs, the functions - - - - int odr_sequence_begin(ODR o, void *p, int size, const char *name); - int odr_sequence_end(ODR o); - - - -are provided. - - - -The odr_sequence_begin() function should be -called in the beginning of a function that implements a SEQUENCE type. -Its parameters are the &odr; stream, a pointer (to a pointer to the type -you're implementing), and the size of the type -(typically a C structure). On encoding, it returns 1 if -* p is a null pointer. The size -parameter is ignored. On decoding, it returns 1 if the type is found in -the data stream. size bytes of memory are allocated, -and *p is set to point to this space. -odr_sequence_end() is called at the end of the -complex function. Assume that a type is defined like this: - - - - MySequence ::= SEQUENCE { - intval INTEGER, - boolval BOOLEAN OPTIONAL } - + + + + The function myInt() can then be used like any of + the primitive functions provided by &odr;. Note that the behavior of + odr_explicit() + and odr_implicit() macros + act exactly the same as the functions they are applied to - they + respond to error conditions, etc, in the same manner - they + simply have three extra parameters. The class parameter may + take one of the values: ODR_CONTEXT, + ODR_PRIVATE, ODR_UNIVERSAL, or + /ODR_APPLICATION. + + + + Constructed Types + + + Constructed types are created by combining primitive types. The + &odr; system only implements the SEQUENCE and SEQUENCE OF constructions + (although adding the rest of the container types should be simple + enough, if the need arises). + + + + For implementing SEQUENCEs, the functions + + + +int odr_sequence_begin(ODR o, void *p, int size, const char *name); +int odr_sequence_end(ODR o); + + + + are provided. + + + + The odr_sequence_begin() function should be + called in the beginning of a function that implements a SEQUENCE type. + Its parameters are the &odr; stream, a pointer (to a pointer to the type + you're implementing), and the size of the type + (typically a C structure). On encoding, it returns 1 if + * p is a null pointer. The size + parameter is ignored. On decoding, it returns 1 if the type is found in + the data stream. size bytes of memory are allocated, + and *p is set to point to this space. + odr_sequence_end() is called at the end of the + complex function. Assume that a type is defined like this: + + + +MySequence ::= SEQUENCE { + intval INTEGER, + boolval BOOLEAN OPTIONAL +} + - -The corresponding &odr; encoder/decoder function and the associated data -structures could be written like this: - + + The corresponding &odr; encoder/decoder function and the associated data + structures could be written like this: + - - typedef struct MySequence - { + +typedef struct MySequence +{ int *intval; bool_t *boolval; - } MySequence; - - int mySequence(ODR o, MySequence **p, int optional, const char *name) - { +} MySequence; + +int mySequence(ODR o, MySequence **p, int optional, const char *name) +{ if (odr_sequence_begin(o, p, sizeof(**p), name) == 0) return optional && odr_ok(o); return odr_integer(o, &(*p)->intval, 0, "intval") && odr_bool(o, &(*p)->boolval, 1, "boolval") && odr_sequence_end(o); - } - - - -Note the 1 in the call to odr_bool(), to mark -that the sequence member is optional. -If either of the member types had been tagged, the macros -odr_implicit() or odr_explicit() -could have been used. -The new function can be used exactly like the standard functions provided -with &odr;. It will encode, decode or pretty-print a data value of the -MySequence type. We like to name types with an -initial capital, as done in ASN.1 definitions, and to name the -corresponding function with the first character of the name in lower case. -You could, of course, name your structures, types, and functions any way -you please - as long as you're consistent, and your code is easily readable. -odr_ok is just that - a predicate that returns the -state of the stream. It is used to ensure that the behaviour of the new -type is compatible with the interface of the primitive types. - - - -Tagging Constructed Types - - - -See section Tagging Primitive types -for information on how to tag the primitive types, as well as types -that are already defined. - - - -Implicit Tagging - - -Assume the type above had been defined as - - - - MySequence ::= [10] IMPLICIT SEQUENCE { - intval INTEGER, - boolval BOOLEAN OPTIONAL } - - - -You would implement this in &odr; by calling the function - - - - int odr_implicit_settag(ODR o, int class, int tag); - - - -which overrides the tag of the type immediately following it. The -macro odr_implicit() works by calling -odr_implicit_settag() immediately -before calling the function pointer argument. -Your type function could look like this: - - - - int mySequence(ODR o, MySequence **p, int optional, const char *name) - { +} + + + + + Note the 1 in the call to odr_bool(), to mark + that the sequence member is optional. + If either of the member types had been tagged, the macros + odr_implicit() or odr_explicit() + could have been used. + The new function can be used exactly like the standard functions provided + with &odr;. It will encode, decode or pretty-print a data value of the + MySequence type. We like to name types with an + initial capital, as done in ASN.1 definitions, and to name the + corresponding function with the first character of the name in lower case. + You could, of course, name your structures, types, and functions any way + you please - as long as you're consistent, and your code is easily readable. + odr_ok is just that - a predicate that returns the + state of the stream. It is used to ensure that the behaviour of the new + type is compatible with the interface of the primitive types. + + + + Tagging Constructed Types + + + + See section Tagging Primitive types + for information on how to tag the primitive types, as well as types + that are already defined. + + + + Implicit Tagging + + + Assume the type above had been defined as + + + +MySequence ::= [10] IMPLICIT SEQUENCE { + intval INTEGER, + boolval BOOLEAN OPTIONAL +} + + + + You would implement this in &odr; by calling the function + + + +int odr_implicit_settag(ODR o, int class, int tag); + + + + which overrides the tag of the type immediately following it. The + macro odr_implicit() works by calling + odr_implicit_settag() immediately + before calling the function pointer argument. + Your type function could look like this: + + + +int mySequence(ODR o, MySequence **p, int optional, const char *name) +{ if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 || odr_sequence_begin(o, p, sizeof(**p), name) == 0) return optional && odr_ok(o); return odr_integer(o, &(*p)->intval, 0, "intval") && odr_bool(o, &(*p)->boolval, 1, "boolval") && - odr_sequence_end(o); + odr_sequence_end(o); } - + - -The definition of the structure MySequence would be -the same. - - + + The definition of the structure MySequence would be + the same. + + -Explicit Tagging + Explicit Tagging - -Explicit tagging of constructed types is a little more complicated, -since you are in effect adding a level of construction to the data. - + + Explicit tagging of constructed types is a little more complicated, + since you are in effect adding a level of construction to the data. + - -Assume the definition: - + + Assume the definition: + - - MySequence ::= [10] IMPLICIT SEQUENCE { - intval INTEGER, - boolval BOOLEAN OPTIONAL } - - - -Since the new type has an extra level of construction, two new functions -are needed to encapsulate the base type: - - - - int odr_constructed_begin(ODR o, void *p, int class, int tag, - const char *name); - - int odr_constructed_end(ODR o); - - - -Assume that the IMPLICIT in the type definition above were replaced -with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which -would be equivalent). The structure definition would look the same, -but the function would look like this: - - - - int mySequence(ODR o, MySequence **p, int optional, const char *name) - { + +MySequence ::= [10] IMPLICIT SEQUENCE { + intval INTEGER, + boolval BOOLEAN OPTIONAL +} + + + + Since the new type has an extra level of construction, two new functions + are needed to encapsulate the base type: + + + + int odr_constructed_begin(ODR o, void *p, int class, int tag, + const char *name); + + int odr_constructed_end(ODR o); + + + + Assume that the IMPLICIT in the type definition above were replaced + with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which + would be equivalent). The structure definition would look the same, + but the function would look like this: + + + +int mySequence(ODR o, MySequence **p, int optional, const char *name) +{ if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0) return optional && odr_ok(o); if (o->direction == ODR_DECODE) *p = odr_malloc(o, sizeof(**p)); if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0) { - *p = 0; /* this is almost certainly a protocol error */ - return 0; + *p = 0; /* this is almost certainly a protocol error */ + return 0; } return odr_integer(o, &(*p)->intval, 0, "intval") && odr_bool(o, &(*p)->boolval, 1, "boolval") && odr_sequence_end(o) && odr_constructed_end(o); - } - - - -Notice that the interface here gets kind of nasty. The reason is -simple: Explicitly tagged, constructed types are fairly rare in -the protocols that we care about, so the -aesthetic annoyance (not to mention the dangers of a cluttered -interface) is less than the time that would be required to develop a -better interface. Nevertheless, it is far from satisfying, and it's a -point that will be worked on in the future. One option for you would -be to simply apply the odr_explicit() macro to -the first function, and not -have to worry about odr_constructed_* yourself. -Incidentally, as you might have guessed, the -odr_sequence_ functions are themselves -implemented using the /odr_constructed_ functions. - - - - -SEQUENCE OF - - -To handle sequences (arrays) of a apecific type, the function - - - - int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional), - void *p, int *num, const char *name); - - - -The fun parameter is a pointer to the decoder/encoder -function of the type. p is a pointer to an array of -pointers to your type. num is the number of elements -in the array. - - - -Assume a type - - - - MyArray ::= SEQUENCE OF INTEGER - - - -The C representation might be - - - - typedef struct MyArray - { +} + + + + Notice that the interface here gets kind of nasty. The reason is + simple: Explicitly tagged, constructed types are fairly rare in + the protocols that we care about, so the + aesthetic annoyance (not to mention the dangers of a cluttered + interface) is less than the time that would be required to develop a + better interface. Nevertheless, it is far from satisfying, and it's a + point that will be worked on in the future. One option for you would + be to simply apply the odr_explicit() macro to + the first function, and not + have to worry about odr_constructed_* yourself. + Incidentally, as you might have guessed, the + odr_sequence_ functions are themselves + implemented using the /odr_constructed_ functions. + + + + + SEQUENCE OF + + + To handle sequences (arrays) of a apecific type, the function + + + +int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional), + void *p, int *num, const char *name); + + + + The fun parameter is a pointer to the decoder/encoder + function of the type. p is a pointer to an array of + pointers to your type. num is the number of elements + in the array. + + + + Assume a type + + + +MyArray ::= SEQUENCE OF INTEGER + + + + The C representation might be + + + +typedef struct MyArray +{ int num_elements; int **elements; - } MyArray; - +} MyArray; + - -And the function might look like - + + And the function might look like + - - int myArray(ODR o, MyArray **p, int optional, const char *name) - { + +int myArray(ODR o, MyArray **p, int optional, const char *name) +{ if (o->direction == ODR_DECODE) *p = odr_malloc(o, sizeof(**p)); if (odr_sequence_of(o, odr_integer, &(*p)->elements, - &(*p)->num_elements, name)) - return 1; + &(*p)->num_elements, name)) + return 1; *p = 0; - return optional && odr_ok(o); - } - - - -CHOICE Types - - -The choice type is used fairly often in some ASN.1 definitions, so -some work has gone into streamlining its interface. - - - -CHOICE types are handled by the function: - - - - int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp, - const char *name); - - - -The arm array is used to describe each of the possible -types that the CHOICE type may assume. Internally in your application, -the CHOICE type is represented as a discriminated union. That is, a C union -accompanied by an integer (or enum) identifying the active 'arm' of -the union. whichp is a pointer to the union -discriminator. When encoding, it is examined to determine the current -type. When decoding, it is set to reference the type that was found in -the input stream. - - - -The Odr_arm type is defined thus: - - - - typedef struct odr_arm - { + return optional && odr_ok(o); +} + + + + CHOICE Types + + + The choice type is used fairly often in some ASN.1 definitions, so + some work has gone into streamlining its interface. + + + + CHOICE types are handled by the function: + + + + int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp, + const char *name); + + + + The arm array is used to describe each of the possible + types that the CHOICE type may assume. Internally in your application, + the CHOICE type is represented as a discriminated union. That is, a + C union accompanied by an integer (or enum) identifying the active + 'arm' of the union. + whichp is a pointer to the union discriminator. + When encoding, it is examined to determine the current type. + When decoding, it is set to reference the type that was found in + the input stream. + + + + The Odr_arm type is defined thus: + + + +typedef struct odr_arm +{ int tagmode; int class; int tag; int which; Odr_fun fun; char *name; - } Odr_arm; - - - -The interpretation of the fields are: - - - -tagmode - Either ODR_IMPLICIT, -ODR_EXPLICIT, or ODR_NONE (-1) to mark -no tagging. - - -which - The value of the discriminator that corresponds to -this CHOICE element. Typically, it will be a #defined constant, or -an enum member. - - -fun - A pointer to a function that implements the type of -the CHOICE member. It may be either a standard &odr; type or a type -defined by yourself. - - -name - Name of tag. - - - - -A handy way to prepare the array for use by the -odr_choice() function is to -define it as a static, initialized array in the beginning of your -decoding/encoding function. Assume the type definition: - - - - MyChoice ::= CHOICE { +} Odr_arm; + + + + The interpretation of the fields are: + + + + tagmode + Either ODR_IMPLICIT, + ODR_EXPLICIT, or ODR_NONE (-1) + to mark no tagging. + + + which + The value of the discriminator that corresponds to + this CHOICE element. Typically, it will be a #defined constant, or + an enum member. + + + fun + A pointer to a function that implements the type of + the CHOICE member. It may be either a standard &odr; type or a type + defined by yourself. + + + name + Name of tag. + + + + + A handy way to prepare the array for use by the + odr_choice() function is to + define it as a static, initialized array in the beginning of your + decoding/encoding function. Assume the type definition: + + + +MyChoice ::= CHOICE { untagged INTEGER, tagged [99] IMPLICIT INTEGER, other BOOLEAN - } - - - -Your C type might look like - - - - typedef struct MyChoice - { - enum - { - MyChoice_untagged, - MyChoice_tagged, - MyChoice_other - } which; - union - { - int *untagged; - int *tagged; - bool_t *other; - } u; - }; - - - -And your function could look like this: - - - +} + + + + Your C type might look like + + + +typedef struct MyChoice +{ + enum + { + MyChoice_untagged, + MyChoice_tagged, + MyChoice_other + } which; + union + { + int *untagged; + int *tagged; + bool_t *other; + } u; +}; + + + + And your function could look like this: + + + int myChoice(ODR o, MyChoice **p, int optional, const char *name) { static Odr_arm arm[] = { - {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"}, - {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer, - "tagged"}, - {-1, -1, -1, MyChoice_other, odr_boolean, "other"}, - {-1, -1, -1, -1, 0} + {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"}, + {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer, + "tagged"}, + {-1, -1, -1, MyChoice_other, odr_boolean, "other"}, + {-1, -1, -1, -1, 0} }; if (o->direction == ODR_DECODE) @@ -1154,71 +1166,86 @@ int myChoice(ODR o, MyChoice **p, int optional, const char *name) if (odr_choice(o, arm, &(*p)->u, &(*p)->which), name) return 1; *p = 0; - return optional && odr_ok(o); + return optional && odr_ok(o); } - - - -In some cases (say, a non-optional choice which is a member of a sequence), -you can "embed" the union and its discriminator in the structure -belonging to the enclosing type, and you won't need to fiddle with -memory allocation to create a separate structure to wrap the -discriminator and union. - - - -The corresponding function is somewhat nicer in the Sun XDR interface. -Most of the complexity of this interface comes from the possibility of -declaring sequence elements (including CHOICEs) optional. - - - -The ASN.1 specifictions naturally requires that each member of a -CHOICE have a distinct tag, so they can be told apart on decoding. -Sometimes it can be useful to define a CHOICE that has multiple types -that share the same tag. You'll need some other mechanism, perhaps -keyed to the context of the CHOICE type. In effect, we would like to -introduce a level of context-sensitiveness to our ASN.1 specification. -When encoding an internal representation, we have no problem, as long -as each CHOICE member has a distinct discriminator value. For -decoding, we need a way to tell the choice function to look for a -specific arm of the table. The function - - - - void odr_choice_bias(ODR o, int what); - - - -provides this functionality. When called, it leaves a notice for the next -call to odr_choice() to be called on the decoding -stream o that only the arm entry with -a which field equal to what -should be tried. - - - -The most important application (perhaps the only one, really) is in -the definition of application-specific EXTERNAL encoders/decoders -which will automatically decode an ANY member given the direct or -indirect reference. - - - - - -Debugging - - -The protocol modules are suffering somewhat from a lack of diagnostic -tools at the moment. Specifically ways to pretty-print PDUs that -aren't recognized by the system. We'll include something to this end -in a not-too-distant release. In the meantime, what we do when we get -packages we don't understand is to compile the ODR module with -ODR_DEBUG defined. This causes the module to dump tracing -information as it processes data units. With this output and the -protocol specification (Z39.50), it is generally fairly easy to see -what goes wrong. - - - + + + + In some cases (say, a non-optional choice which is a member of a + sequence), you can "embed" the union and its discriminator in the +structure + belonging to the enclosing type, and you won't need to fiddle with + memory allocation to create a separate structure to wrap the + discriminator and union. + + + + The corresponding function is somewhat nicer in the Sun XDR interface. + Most of the complexity of this interface comes from the possibility of + declaring sequence elements (including CHOICEs) optional. + + + + The ASN.1 specifictions naturally requires that each member of a + CHOICE have a distinct tag, so they can be told apart on decoding. + Sometimes it can be useful to define a CHOICE that has multiple types + that share the same tag. You'll need some other mechanism, perhaps + keyed to the context of the CHOICE type. In effect, we would like to + introduce a level of context-sensitiveness to our ASN.1 specification. + When encoding an internal representation, we have no problem, as long + as each CHOICE member has a distinct discriminator value. For + decoding, we need a way to tell the choice function to look for a + specific arm of the table. The function + + + + void odr_choice_bias(ODR o, int what); + + + + provides this functionality. When called, it leaves a notice for the next + call to odr_choice() to be called on the decoding + stream o that only the arm entry with + a which field equal to what + should be tried. + + + + The most important application (perhaps the only one, really) is in + the definition of application-specific EXTERNAL encoders/decoders + which will automatically decode an ANY member given the direct or + indirect reference. + + + + + + Debugging + + + The protocol modules are suffering somewhat from a lack of diagnostic + tools at the moment. Specifically ways to pretty-print PDUs that + aren't recognized by the system. We'll include something to this end + in a not-too-distant release. In the meantime, what we do when we get + packages we don't understand is to compile the ODR module with + ODR_DEBUG defined. This causes the module to dump tracing + information as it processes data units. With this output and the + protocol specification (Z39.50), it is generally fairly easy to see + what goes wrong. + + + + diff --git a/doc/tools.xml b/doc/tools.xml index 8b3fe80..4b2a9a1 100644 --- a/doc/tools.xml +++ b/doc/tools.xml @@ -1,421 +1,432 @@ - -Supporting Tools - - -In support of the service API - primarily the ASN module, which -provides the programmatic interface to the Z39.50 APDUs, YAZ contains -a collection of tools that support the development of applications. - - -Query Syntax Parsers - - -Since the type-1 (RPN) query structure has no direct, useful string -representation, every origin application needs to provide some form of -mapping from a local query notation or representation to a -Z_RPNQuery structure. Some programmers will prefer to -construct the query manually, perhaps using odr_malloc() -to simplify memory management. The &yaz; distribution includes two separate, -query-generating tools that may be of use to you. - - -Prefix Query Format - - -Since RPN or reverse polish notation is really just a fancy way of -describing a suffix notation format (operator follows operands), it -would seem that the confusion is total when we now introduce a prefix -notation for RPN. The reason is one of simple laziness - it's somewhat -simpler to interpret a prefix format, and this utility was designed -for maximum simplicity, to provide a baseline representation for use -in simple test applications and scripting environments (like Tcl). The -demonstration client included with YAZ uses the PQF. - - -The PQF is defined by the pquery module in the YAZ library. The -pquery.h file provides the declaration of the functions - - + + Supporting Tools + + + In support of the service API - primarily the ASN module, which + provides the programmatic interface to the Z39.50 APDUs, YAZ contains + a collection of tools that support the development of applications. + + + Query Syntax Parsers + + + Since the type-1 (RPN) query structure has no direct, useful string + representation, every origin application needs to provide some form of + mapping from a local query notation or representation to a + Z_RPNQuery structure. Some programmers will prefer to + construct the query manually, perhaps using + odr_malloc() to simplify memory management. + The &yaz; distribution includes two separate, query-generating tools + that may be of use to you. + + + Prefix Query Format + + + Since RPN or reverse polish notation is really just a fancy way of + describing a suffix notation format (operator follows operands), it + would seem that the confusion is total when we now introduce a prefix + notation for RPN. The reason is one of simple laziness - it's somewhat + simpler to interpret a prefix format, and this utility was designed + for maximum simplicity, to provide a baseline representation for use + in simple test applications and scripting environments (like Tcl). The + demonstration client included with YAZ uses the PQF. + + + The PQF is defined by the pquery module in the YAZ library. The + pquery.h file provides the declaration of the + functions + + Z_RPNQuery *p_query_rpn (ODR o, oid_proto proto, const char *qbuf); Z_AttributesPlusTerm *p_query_scan (ODR o, oid_proto proto, - Odr_oid **attributeSetP, const char *qbuf); + Odr_oid **attributeSetP, const char *qbuf); int p_query_attset (const char *arg); - - -The function p_query_rpn() takes as arguments an -&odr; stream (see section The ODR Module) -to provide a memory source (the structure created is released on -the next call to odr_reset() on the stream), a -protocol identifier (one of the constants PROTO_Z3950 and -PROTO_SR), an attribute set -reference, and finally a null-terminated string holding the query -string. - - -If the parse went well, p_query_rpn() returns a -pointer to a Z_RPNQuery structure which can be -placed directly into a Z_SearchRequest. - - + + + The function p_query_rpn() takes as arguments an + &odr; stream (see section The ODR Module) + to provide a memory source (the structure created is released on + the next call to odr_reset() on the stream), a + protocol identifier (one of the constants PROTO_Z3950 and + PROTO_SR), an attribute set + reference, and finally a null-terminated string holding the query + string. + + + If the parse went well, p_query_rpn() returns a + pointer to a Z_RPNQuery structure which can be + placed directly into a Z_SearchRequest. + + -The p_query_attset specifies which attribute set to use if -the query doesn't specify one by the @attrset operator. -The p_query_attset returns 0 if the argument is a -valid attribute set specifier; otherwise the function returns -1. - + The p_query_attset specifies which attribute set + to use if the query doesn't specify one by the + @attrset operator. + The p_query_attset returns 0 if the argument is a + valid attribute set specifier; otherwise the function returns -1. + - -The grammar of the PQF is as follows: - + + The grammar of the PQF is as follows: + - -Query ::= [ AttSet ] QueryStruct. + + Query ::= [ AttSet ] QueryStruct. -AttSet ::= string. + AttSet ::= string. -QueryStruct ::= { Attribute } Simple | Complex. + QueryStruct ::= { Attribute } Simple | Complex. -Attribute ::= '@attr' AttributeType '=' AttributeValue. + Attribute ::= '@attr' AttributeType '=' AttributeValue. -AttributeType ::= integer. + AttributeType ::= integer. -AttributeValue ::= integer. + AttributeValue ::= integer. -Complex ::= Operator QueryStruct QueryStruct. + Complex ::= Operator QueryStruct QueryStruct. -Operator ::= '@and' | '@or' | '@not' | '@prox' Proximity. + Operator ::= '@and' | '@or' | '@not' | '@prox' Proximity. -Simple ::= ResultSet | Term. + Simple ::= ResultSet | Term. -ResultSet ::= '@set' string. + ResultSet ::= '@set' string. -Term ::= string | '"' string '"'. + Term ::= string | '"' string '"'. -Proximity ::= Exclusion Distance Ordered Relation WhichCode UnitCode. + Proximity ::= Exclusion Distance Ordered Relation WhichCode UnitCode. -Exclusion ::= '1' | '0' | 'void'. + Exclusion ::= '1' | '0' | 'void'. -Distance ::= integer. + Distance ::= integer. -Ordered ::= '1' | '0'. + Ordered ::= '1' | '0'. -Relation ::= integer. + Relation ::= integer. -WhichCode ::= 'known' | 'private' | integer. + WhichCode ::= 'known' | 'private' | integer. -UnitCode ::= integer. - + UnitCode ::= integer. + - -You will note that the syntax above is a fairly faithful -representation of RPN, except for the Attibute, which has been -moved a step away from the term, allowing you to associate one or more -attributes with an entire query structure. The parser will -automatically apply the given attributes to each term as required. - + + You will note that the syntax above is a fairly faithful + representation of RPN, except for the Attibute, which has been + moved a step away from the term, allowing you to associate one or more + attributes with an entire query structure. The parser will + automatically apply the given attributes to each term as required. + - -The following are all examples of valid queries in the PQF. - + + The following are all examples of valid queries in the PQF. + - -dylan + + dylan -"bob dylan" + "bob dylan" -@or "dylan" "zimmerman" + @or "dylan" "zimmerman" -@set Result-1 + @set Result-1 -@or @and bob dylan @set Result-1 + @or @and bob dylan @set Result-1 -@attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming" + @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming" -@attr 4=1 @attr 1=4 "self portrait" + @attr 4=1 @attr 1=4 "self portrait" -@prox 0 3 1 2 k 2 dylan zimmerman - + @prox 0 3 1 2 k 2 dylan zimmerman + - -Common Command Language + + Common Command Language - -Not all users enjoy typing in prefix query structures and numerical -attribute values, even in a minimalistic test client. In the library -world, the more intuitive Common Command Language (or ISO 8777) has -enjoyed some popularity - especially before the widespread -availability of graphical interfaces. It is still useful in -applications where you for some reason or other need to provide a -symbolic language for expressing boolean query structures. - + + Not all users enjoy typing in prefix query structures and numerical + attribute values, even in a minimalistic test client. In the library + world, the more intuitive Common Command Language (or ISO 8777) has + enjoyed some popularity - especially before the widespread + availability of graphical interfaces. It is still useful in + applications where you for some reason or other need to provide a + symbolic language for expressing boolean query structures. + - -The EUROPAGATE research project working under the Libraries programme -of the European Commission's DG XIII has, amongst other useful tools, -implemented a general-purpose CCL parser which produces an output -structure that can be trivially converted to the internal RPN -representation of YAZ (The Z_RPNQuery structure). -Since the CCL utility - along with the rest of the software -produced by EUROPAGATE - is made freely available on a liberal license, it -is included as a supplement to YAZ. - + + The EUROPAGATE research project working under the Libraries programme + of the European Commission's DG XIII has, amongst other useful tools, + implemented a general-purpose CCL parser which produces an output + structure that can be trivially converted to the internal RPN + representation of YAZ (The Z_RPNQuery structure). + Since the CCL utility - along with the rest of the software + produced by EUROPAGATE - is made freely available on a liberal license, it + is included as a supplement to YAZ. + -CCL Syntax + CCL Syntax - -The CCL parser obeys the following grammar for the FIND argument. -The syntax is annotated by in the lines prefixed by -‐‐. - + + The CCL parser obeys the following grammar for the FIND argument. + The syntax is annotated by in the lines prefixed by + ‐‐. + - -CCL-Find ::= CCL-Find Op Elements - | Elements. + + CCL-Find ::= CCL-Find Op Elements + | Elements. -Op ::= "and" | "or" | "not" --- The above means that Elements are separated by boolean operators. + Op ::= "and" | "or" | "not" + -- The above means that Elements are separated by boolean operators. -Elements ::= '(' CCL-Find ')' - | Set - | Terms - | Qualifiers Relation Terms - | Qualifiers Relation '(' CCL-Find ')' - | Qualifiers '=' string '-' string --- Elements is either a recursive definition, a result set reference, a --- list of terms, qualifiers followed by terms, qualifiers followed --- by a recursive definition or qualifiers in a range (lower - upper). + Elements ::= '(' CCL-Find ')' + | Set + | Terms + | Qualifiers Relation Terms + | Qualifiers Relation '(' CCL-Find ')' + | Qualifiers '=' string '-' string + -- Elements is either a recursive definition, a result set reference, a + -- list of terms, qualifiers followed by terms, qualifiers followed + -- by a recursive definition or qualifiers in a range (lower - upper). -Set ::= 'set' = string --- Reference to a result set + Set ::= 'set' = string + -- Reference to a result set -Terms ::= Terms Prox Term - | Term --- Proximity of terms. - -Term ::= Term string - | string --- This basically means that a term may include a blank - -Qualifiers ::= Qualifiers ',' string - | string --- Qualifiers is a list of strings separated by comma - -Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<' --- Relational operators. This really doesn't follow the ISO8777 --- standard. - -Prox ::= '%' | '!' --- Proximity operator - - - - -The following queries are all valid: - - - -dylan - -"bob dylan" - -dylan or zimmerman - -set=1 - -(dylan and bob) or set=1 - - - -Assuming that the qualifiers ti, au -and date are defined we may use: - - - -ti=self portrait - -au=(bob dylan and slow train coming) - -date>1980 and (ti=((self portrait))) - - - - -CCL Qualifiers - - -Qualifiers are used to direct the search to a particular searchable -index, such as title (ti) and author indexes (au). The CCL standard -itself doesn't specify a particular set of qualifiers, but it does -suggest a few short-hand notations. You can customize the CCL parser -to support a particular set of qualifiers to relect the current target -profile. Traditionally, a qualifier would map to a particular -use-attribute within the BIB-1 attribute set. However, you could also -define qualifiers that would set, for example, the -structure-attribute. - - - -Consider a scenario where the target support ranked searches in the -title-index. In this case, the user could specify - - -> -ti,ranked=knuth computer - - -and the ranked would map to structure=free-form-text -(4=105) and the ti would map to title (1=4). - - - -A "profile" with a set predefined CCL qualifiers can be read from a -file. The YAZ client reads its CCL qualifiers from a file named -default.bib. Each line in the file has the form: - - - -qualifier-name - type=val type=val ... - - - -where qualifier-name is the name of the -qualifier to be used (eg. ti), -type is a BIB-1 category type and -val is the corresponding BIB-1 attribute value. -The type can be either numeric or it may be -either u (use), r (relation), -p (position), s (structure), -t (truncation) or c (completeness). -The qualifier-name term has a -special meaning. The types and values for this definition is used when -no qualifiers are present. - - - -Consider the following definition: - - - -ti u=4 s=1 -au u=1 s=1 -term s=105 - - -Two qualifiers are defined, ti and au. -They both set the structure-attribute to phrase (1). ti -sets the use-attribute to 4. au sets the use-attribute -to 1. When no qualifiers are used in the query the structure-attribute is -set to free-form-text (105). - - - -CCL API - -All public definitions can be found in the header file -ccl.h. A profile identifier is of type -CCL_bibset. A profile must be created with the call to -the function ccl_qual_mk which returns a profile -handle of type CCL_bibset. - - - -To read a file containing qualifier definitions the function -ccl_qual_file may be convenient. This function takes -an already opened FILE handle pointer as argument -along with a CCL_bibset handle. - - - -To parse a simple string with a FIND query use the function - - - struct ccl_rpn_node *ccl_find_str (CCL_bibset bibset, const char *str, - int *error, int *pos); - - -which takes the CCL profile (bibset) and query -(str) as input. Upon successful completion the RPN -tree is returned. If an error eccur, such as a syntax error, the integer -pointed to by error holds the error code and -pos holds the offset inside query string in which -the parsing failed. - - - -An english representation of the error may be obtained by calling -the ccl_err_msg function. The error codes are listed in -ccl.h. - - - -To convert the CCL RPN tree (type struct ccl_rpn_node *) -to the Z_RPNQuery of YAZ the function ccl_rpn_query -must be used. This function which is part of YAZ is implemented in -yaz-ccl.c. -After calling this function the CCL RPN tree is probably no longer -needed. The ccl_rpn_delete destroys the CCL RPN tree. - - - -A CCL profile may be destroyed by calling the ccl_qual_rm -function. - - - -The token names for the CCL operators may be changed by setting the -globals (all type char *) -ccl_token_and, ccl_token_or, -ccl_token_not and ccl_token_set. -An operator may have aliases, i.e. there may be more than one name for -the operator. To do this, separate each alias with a space character. - - - - -Object Identifiers - - -The basic YAZ representation of an OID is an array of integers, -terminated with the value -1. The &odr; module provides two -utility-functions to create and copy this type of data elements: - - - - Odr_oid *odr_getoidbystr(ODR o, char *str); - - - -Creates an OID based on a string-based representation using dots (.) -to separate elements in the OID. - - - -Odr_oid *odr_oiddup(ODR odr, Odr_oid *o); - - - -Creates a copy of the OID referenced by the o parameter. -Both functions take an &odr; stream as parameter. This stream is used to -allocate memory for the data elements, which is released on a -subsequent call to odr_reset() on that stream. - - - -The OID module provides a higher-level representation of the -family of object identifers which describe the Z39.50 protocol and its -related objects. The definition of the module interface is given in -the oid.h file. - - - -The interface is mainly based on the oident structure. The -definition of this structure looks like this: - - - + Terms ::= Terms Prox Term + | Term + -- Proximity of terms. + + Term ::= Term string + | string + -- This basically means that a term may include a blank + + Qualifiers ::= Qualifiers ',' string + | string + -- Qualifiers is a list of strings separated by comma + + Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<' + -- Relational operators. This really doesn't follow the ISO8777 + -- standard. + + Prox ::= '%' | '!' + -- Proximity operator + + + + + The following queries are all valid: + + + + dylan + + "bob dylan" + + dylan or zimmerman + + set=1 + + (dylan and bob) or set=1 + + + + Assuming that the qualifiers ti, au + and date are defined we may use: + + + + ti=self portrait + + au=(bob dylan and slow train coming) + + date>1980 and (ti=((self portrait))) + + + + + CCL Qualifiers + + + Qualifiers are used to direct the search to a particular searchable + index, such as title (ti) and author indexes (au). The CCL standard + itself doesn't specify a particular set of qualifiers, but it does + suggest a few short-hand notations. You can customize the CCL parser + to support a particular set of qualifiers to relect the current target + profile. Traditionally, a qualifier would map to a particular + use-attribute within the BIB-1 attribute set. However, you could also + define qualifiers that would set, for example, the + structure-attribute. + + + + Consider a scenario where the target support ranked searches in the + title-index. In this case, the user could specify + + + > + ti,ranked=knuth computer + + + and the ranked would map to structure=free-form-text + (4=105) and the ti would map to title (1=4). + + + + A "profile" with a set predefined CCL qualifiers can be read from a + file. The YAZ client reads its CCL qualifiers from a file named + default.bib. Each line in the file has the form: + + + + qualifier-name + type=val + type=val ... + + + + where qualifier-name is the name of the + qualifier to be used (eg. ti), + type is a BIB-1 category type and + val is the corresponding BIB-1 attribute + value. + The type can be either numeric or it may be + either u (use), r (relation), + p (position), s (structure), + t (truncation) or c (completeness). + The qualifier-name term + has a special meaning. + The types and values for this definition is used when + no qualifiers are present. + + + + Consider the following definition: + + + + ti u=4 s=1 + au u=1 s=1 + term s=105 + + + Two qualifiers are defined, ti and + au. + They both set the structure-attribute to phrase (1). + ti + sets the use-attribute to 4. au sets the + use-attribute to 1. + When no qualifiers are used in the query the structure-attribute is + set to free-form-text (105). + + + + CCL API + + All public definitions can be found in the header file + ccl.h. A profile identifier is of type + CCL_bibset. A profile must be created with the call + to the function ccl_qual_mk which returns a profile + handle of type CCL_bibset. + + + + To read a file containing qualifier definitions the function + ccl_qual_file may be convenient. This function + takes an already opened FILE handle pointer as + argument along with a CCL_bibset handle. + + + + To parse a simple string with a FIND query use the function + + + struct ccl_rpn_node *ccl_find_str (CCL_bibset bibset, const char *str, + int *error, int *pos); + + + which takes the CCL profile (bibset) and query + (str) as input. Upon successful completion the RPN + tree is returned. If an error eccur, such as a syntax error, the integer + pointed to by error holds the error code and + pos holds the offset inside query string in which + the parsing failed. + + + + An english representation of the error may be obtained by calling + the ccl_err_msg function. The error codes are + listed in ccl.h. + + + + To convert the CCL RPN tree (type + struct ccl_rpn_node *) + to the Z_RPNQuery of YAZ the function ccl_rpn_query + must be used. This function which is part of YAZ is implemented in + yaz-ccl.c. + After calling this function the CCL RPN tree is probably no longer + needed. The ccl_rpn_delete destroys the CCL RPN tree. + + + + A CCL profile may be destroyed by calling the + ccl_qual_rm function. + + + + The token names for the CCL operators may be changed by setting the + globals (all type char *) + ccl_token_and, ccl_token_or, + ccl_token_not and ccl_token_set. + An operator may have aliases, i.e. there may be more than one name for + the operator. To do this, separate each alias with a space character. + + + + + Object Identifiers + + + The basic YAZ representation of an OID is an array of integers, + terminated with the value -1. The &odr; module provides two + utility-functions to create and copy this type of data elements: + + + + Odr_oid *odr_getoidbystr(ODR o, char *str); + + + + Creates an OID based on a string-based representation using dots (.) + to separate elements in the OID. + + + + Odr_oid *odr_oiddup(ODR odr, Odr_oid *o); + + + + Creates a copy of the OID referenced by the o + parameter. + Both functions take an &odr; stream as parameter. This stream is used to + allocate memory for the data elements, which is released on a + subsequent call to odr_reset() on that stream. + + + + The OID module provides a higher-level representation of the + family of object identifers which describe the Z39.50 protocol and its + related objects. The definition of the module interface is given in + the oid.h file. + + + + The interface is mainly based on the oident structure. + The definition of this structure looks like this: + + + typedef struct oident { oid_proto proto; @@ -424,252 +435,269 @@ typedef struct oident int oidsuffix[OID_SIZE]; char *desc; } oident; - - - -The proto field takes one of the values - - - -PROTO_Z3950 -PROTO_SR - - - -If you don't care about talking to SR-based implementations (few -exist, and they may become fewer still if and when the ISO SR and ANSI -Z39.50 documents are merged into a single standard), you can ignore -this field on incoming packages, and always set it to PROTO_Z3950 -for outgoing packages. - - - -The oclass field takes one of the values - - - -CLASS_APPCTX -CLASS_ABSYN -CLASS_ATTSET -CLASS_TRANSYN -CLASS_DIAGSET -CLASS_RECSYN -CLASS_RESFORM -CLASS_ACCFORM -CLASS_EXTSERV -CLASS_USERINFO -CLASS_ELEMSPEC -CLASS_VARSET -CLASS_SCHEMA -CLASS_TAGSET -CLASS_GENERAL - - - -corresponding to the OID classes defined by the Z39.50 standard. - -Finally, the value field takes one of the values - - - -VAL_APDU -VAL_BER -VAL_BASIC_CTX -VAL_BIB1 -VAL_EXP1 -VAL_EXT1 -VAL_CCL1 -VAL_GILS -VAL_WAIS -VAL_STAS -VAL_DIAG1 -VAL_ISO2709 -VAL_UNIMARC -VAL_INTERMARC -VAL_CCF -VAL_USMARC -VAL_UKMARC -VAL_NORMARC -VAL_LIBRISMARC -VAL_DANMARC -VAL_FINMARC -VAL_MAB -VAL_CANMARC -VAL_SBN -VAL_PICAMARC -VAL_AUSMARC -VAL_IBERMARC -VAL_EXPLAIN -VAL_SUTRS -VAL_OPAC -VAL_SUMMARY -VAL_GRS0 -VAL_GRS1 -VAL_EXTENDED -VAL_RESOURCE1 -VAL_RESOURCE2 -VAL_PROMPT1 -VAL_DES1 -VAL_KRB1 -VAL_PRESSET -VAL_PQUERY -VAL_PCQUERY -VAL_ITEMORDER -VAL_DBUPDATE -VAL_EXPORTSPEC -VAL_EXPORTINV -VAL_NONE -VAL_SETM -VAL_SETG -VAL_VAR1 -VAL_ESPEC1 - - - -again, corresponding to the specific OIDs defined by the standard. - - - -The desc field contains a brief, mnemonic name for the OID in question. - - - -The function - - - - struct oident *oid_getentbyoid(int *o); - - - -takes as argument an OID, and returns a pointer to a static area -containing an oident structure. You typically use -this function when you receive a PDU containing an OID, and you wish -to branch out depending on the specific OID value. - - - -The function - - - - int *oid_ent_to_oid(struct oident *ent, int *dst); - - - -Takes as argument an oident structure - in which -the proto, oclass/, and -value fields are assumed to be set correctly - -and returns a pointer to a the buffer as given by dst -containing the base -representation of the corresponding OID. The function returns -NULL and the array dst is unchanged if a mapping couldn't place. -The array dst should be at least of size -OID_SIZE. - - - -The oid_ent_to_oid() function can be used whenever -you need to prepare a PDU containing one or more OIDs. The separation of -the protocol element from the remainer of the -OID-description makes it simple to write applications that can -communicate with either Z39.50 or OSI SR-based applications. - - - -The function - - -< - oid_value oid_getvalbyname(const char *name); - - - -takes as argument a mnemonic OID name, and returns the -/value field of the first entry in the database that -contains the given name in its desc field. - - - -Finally, the module provides the following utility functions, whose -meaning should be obvious: - - - - void oid_oidcpy(int *t, int *s); - void oid_oidcat(int *t, int *s); - int oid_oidcmp(int *o1, int *o2); - int oid_oidlen(int *o); - - - - -The OID module has been criticized - and perhaps rightly so -- for needlessly abstracting the -representation of OIDs. Other toolkits use a simple -string-representation of OIDs with good results. In practice, we have -found the interface comfortable and quick to work with, and it is a -simple matter (for what it's worth) to create applications compatible with -both ISO SR and Z39.50. Finally, the use of the /oident -database is by no means mandatory. You can easily create your -own system for representing OIDs, as long as it is compatible with the -low-level integer-array representation of the ODR module. - - - - - -Nibble Memory - - -Sometimes when you need to allocate and construct a large, -interconnected complex of structures, it can be a bit of a pain to -release the associated memory again. For the structures describing the -Z39.50 PDUs and related structures, it is convenient to use the -memory-management system of the &odr; subsystem (see -Using ODR). However, in some circumstances -where you might otherwise benefit from using a simple nibble memory -management system, it may be impractical to use -odr_malloc() and odr_reset(). -For this purpose, the memory manager which also supports the &odr; streams -is made available in the NMEM module. The external interface to this module is given in the nmem.h file. - - - -The following prototypes are given: - - - -NMEM nmem_create(void); -void nmem_destroy(NMEM n); -void *nmem_malloc(NMEM n, int size); -void nmem_reset(NMEM n); -int nmem_total(NMEM n); -void nmem_init(void); - - - -The nmem_create() function returns a pointer to a -memory control handle, which can be released again by -nmem_destroy() when no longer needed. -The function nmem_malloc() allocates a block of -memory of the requested size. A call to nmem_reset() or -nmem_destroy() will release all memory allocated on -the handle since it was created (or since the last call to -nmem_reset(). The function -nmem_total() returns the number of bytes currently -allocated on the handle. - - - - -The nibble memory pool is shared amonst threads. POSIX -mutex'es and WIN32 Critical sections are introduced to keep the -module thread safe. On WIN32 function nmem_init() -initialises the Critical Section handle and should be called once before any -other nmem function is used. - - - - - \ No newline at end of file + + + + The proto field takes one of the values + + + + PROTO_Z3950 + PROTO_SR + + + + If you don't care about talking to SR-based implementations (few + exist, and they may become fewer still if and when the ISO SR and ANSI + Z39.50 documents are merged into a single standard), you can ignore + this field on incoming packages, and always set it to PROTO_Z3950 + for outgoing packages. + + + + The oclass field takes one of the values + + + + CLASS_APPCTX + CLASS_ABSYN + CLASS_ATTSET + CLASS_TRANSYN + CLASS_DIAGSET + CLASS_RECSYN + CLASS_RESFORM + CLASS_ACCFORM + CLASS_EXTSERV + CLASS_USERINFO + CLASS_ELEMSPEC + CLASS_VARSET + CLASS_SCHEMA + CLASS_TAGSET + CLASS_GENERAL + + + + corresponding to the OID classes defined by the Z39.50 standard. + + Finally, the value field takes one of the values + + + + VAL_APDU + VAL_BER + VAL_BASIC_CTX + VAL_BIB1 + VAL_EXP1 + VAL_EXT1 + VAL_CCL1 + VAL_GILS + VAL_WAIS + VAL_STAS + VAL_DIAG1 + VAL_ISO2709 + VAL_UNIMARC + VAL_INTERMARC + VAL_CCF + VAL_USMARC + VAL_UKMARC + VAL_NORMARC + VAL_LIBRISMARC + VAL_DANMARC + VAL_FINMARC + VAL_MAB + VAL_CANMARC + VAL_SBN + VAL_PICAMARC + VAL_AUSMARC + VAL_IBERMARC + VAL_EXPLAIN + VAL_SUTRS + VAL_OPAC + VAL_SUMMARY + VAL_GRS0 + VAL_GRS1 + VAL_EXTENDED + VAL_RESOURCE1 + VAL_RESOURCE2 + VAL_PROMPT1 + VAL_DES1 + VAL_KRB1 + VAL_PRESSET + VAL_PQUERY + VAL_PCQUERY + VAL_ITEMORDER + VAL_DBUPDATE + VAL_EXPORTSPEC + VAL_EXPORTINV + VAL_NONE + VAL_SETM + VAL_SETG + VAL_VAR1 + VAL_ESPEC1 + + + + again, corresponding to the specific OIDs defined by the standard. + + + + The desc field contains a brief, mnemonic name for the OID in question. + + + + The function + + + + struct oident *oid_getentbyoid(int *o); + + + + takes as argument an OID, and returns a pointer to a static area + containing an oident structure. You typically use + this function when you receive a PDU containing an OID, and you wish + to branch out depending on the specific OID value. + + + + The function + + + + int *oid_ent_to_oid(struct oident *ent, int *dst); + + + + Takes as argument an oident structure - in which + the proto, oclass/, and + value fields are assumed to be set correctly - + and returns a pointer to a the buffer as given by dst + containing the base + representation of the corresponding OID. The function returns + NULL and the array dst is unchanged if a mapping couldn't place. + The array dst should be at least of size + OID_SIZE. + + + + The oid_ent_to_oid() function can be used whenever + you need to prepare a PDU containing one or more OIDs. The separation of + the protocol element from the remainer of the + OID-description makes it simple to write applications that can + communicate with either Z39.50 or OSI SR-based applications. + + + + The function + + + < + oid_value oid_getvalbyname(const char *name); + + + + takes as argument a mnemonic OID name, and returns the + /value field of the first entry in the database that + contains the given name in its desc field. + + + + Finally, the module provides the following utility functions, whose + meaning should be obvious: + + + + void oid_oidcpy(int *t, int *s); + void oid_oidcat(int *t, int *s); + int oid_oidcmp(int *o1, int *o2); + int oid_oidlen(int *o); + + + + + The OID module has been criticized - and perhaps rightly so + - for needlessly abstracting the + representation of OIDs. Other toolkits use a simple + string-representation of OIDs with good results. In practice, we have + found the interface comfortable and quick to work with, and it is a + simple matter (for what it's worth) to create applications compatible + with both ISO SR and Z39.50. Finally, the use of the + /oident database is by no means mandatory. + You can easily create your own system for representing OIDs, as long + as it is compatible with the low-level integer-array representation + of the ODR module. + + + + + + Nibble Memory + + + Sometimes when you need to allocate and construct a large, + interconnected complex of structures, it can be a bit of a pain to + release the associated memory again. For the structures describing the + Z39.50 PDUs and related structures, it is convenient to use the + memory-management system of the &odr; subsystem (see + Using ODR). However, in some circumstances + where you might otherwise benefit from using a simple nibble memory + management system, it may be impractical to use + odr_malloc() and odr_reset(). + For this purpose, the memory manager which also supports the &odr; + streams is made available in the NMEM module. The external interface + to this module is given in the nmem.h file. + + + + The following prototypes are given: + + + + NMEM nmem_create(void); + void nmem_destroy(NMEM n); + void *nmem_malloc(NMEM n, int size); + void nmem_reset(NMEM n); + int nmem_total(NMEM n); + void nmem_init(void); + + + + The nmem_create() function returns a pointer to a + memory control handle, which can be released again by + nmem_destroy() when no longer needed. + The function nmem_malloc() allocates a block of + memory of the requested size. A call to nmem_reset() + or nmem_destroy() will release all memory allocated + on the handle since it was created (or since the last call to + nmem_reset(). The function + nmem_total() returns the number of bytes currently + allocated on the handle. + + + + + The nibble memory pool is shared amonst threads. POSIX + mutex'es and WIN32 Critical sections are introduced to keep the + module thread safe. On WIN32 function nmem_init() + initialises the Critical Section handle and should be called once + before any other nmem function is used. + + + + + + + diff --git a/doc/yaz.xml b/doc/yaz.xml index 1710087..6b50fcf 100644 --- a/doc/yaz.xml +++ b/doc/yaz.xml @@ -16,42 +16,42 @@ ODR"> COMSTACK"> ]> - + - -YAZ User's Guide and Reference -SebastianHammer -AdamDickmeiss - -1995 -1996 -1997 -1998 -1999 -2000 -2001 -Index Data - - -This document is the programmer's guide and reference to the &yaz; -package. &yaz; is a compact toolkit that provides access to the -Z39.50 protocol, as well as a set of higher-level tools for -implementing the server and client roles, respectively. -The documentation can be used on its own, or as a reference when looking -at the example applications provided with the package. - - - -&chap-introduction; -&chap-installation; -&chap-asn; -&chap-tools; -&chap-odr; -&chap-comstack; -&chap-frontend; -&chap-future; -&app-license; -&app-indexdata; + + YAZ User's Guide and Reference + SebastianHammer + AdamDickmeiss + + 1995 + 1996 + 1997 + 1998 + 1999 + 2000 + 2001 + Index Data + + + This document is the programmer's guide and reference to the &yaz; + package. &yaz; is a compact toolkit that provides access to the + Z39.50 protocol, as well as a set of higher-level tools for + implementing the server and client roles, respectively. + The documentation can be used on its own, or as a reference when looking + at the example applications provided with the package. + + + + &chap-introduction; + &chap-installation; + &chap-asn; + &chap-tools; + &chap-odr; + &chap-comstack; + &chap-frontend; + &chap-future; + &app-license; + &app-indexdata;