1 <!doctype linuxdoc system>
3 <title>YAZ User's Guide and Reference
4 <author><htmlurl url="http://www.indexdata.dk/" name="Index Data">, <tt><htmlurl url="mailto:info@indexdata.dk" name="info@indexdata.dk"></>
7 This document is the programmer's guide and reference to the YAZ
8 package. YAZ is a compact toolkit that provides access to the
9 Z39.50/SR protocol, as well as a set of higher-level tools for
10 implementing the server and client roles, respectively.
11 The documentation can be used on its own, or as a reference when looking
12 at the example applications provided with the package.
20 The <bf/YAZ/ toolkit offers several different levels of access to the
21 Z39.50 and SR protocols. The level that you need to use depends on
22 your requirements, and the role (server or client) that you
25 The basic level, which is independent of the role, consists of three
29 <item><bf/ASN/, which provides a C representation of the Z39.50/SR protocol
31 <item><bf/ODR/, which encodes and decodes the packages according to the BER
33 <item><bf/COMSTACK/, which exchanges the encoded packages with a peer process
37 The ASN module represents the ASN.1 definition of the
38 SR/Z39.50 protocol. It establishes a set of type and
39 structure definitions, with one structure for each of the top-level
40 PDUs, and one structure or type for each of the contained ASN.1 types.
41 For primitive types, or other types that are defined by the ASN.1
42 standard itself (such as the EXTERNAL type), the C representation is provided
43 by the <bf/ODR/ (Open Data Representation) subsystem.
45 <bf/ODR/ is a basic mechanism for representing an
46 ASN.1 type in the C programming language, and for implementing BER
47 encoders and decoders for values of that type. The types defined in
49 module generally have the prefix <tt/Z_/, and a suffix corresponding to
50 the name of the type in the ASN.1
51 specification of the protocol (generally Z39.50-1995). In the case of
52 base types (those originating in the ASN.1 standard itself), the prefix
53 <tt/Odr_/ is sometimes seen. Either way, look for
54 the actual definition in either <tt/proto.h/ (for the types from the
55 protocol), <tt/odr.h/ (for the primitive ASN.1 types, or <tt/odr_use.h/ (for the
56 ASN.1 <it/useful/ types). The <bf/ASN/ library also provides functions (which
57 are, in turn, defined using <bf/ODR/ primitives) for encoding and decoding data
58 values. Their general form is
61 int z_xxx(ODR o, Z_xxx **p, int optional, const char *name);
64 (note the lower-case &dquot;z&dquot; in the function name)
67 NOTE: If you are using the premade definitions of the <bf/ASN/ module, and
68 you are not adding new protocol of your own, the only parts of ODR
69 that you need to worry about are documented in section
70 <ref id="odr-use" name="Using ODR">.
73 When you have created a BER-encoded buffer, you can use the <bf/COMSTACK/
74 subsystem to transmit (or receive) data over the network. The <bf/COMSTACK/
75 module provides simple functions for establishing a connection
76 (passively or actively, depending on the role of your application),
77 and for exchanging BER-encoded PDUs over that connection. When you
78 create a connection endpoint, you need to specify what transport to
79 use (OSI or TCP/IP), and which protocol you want to use (SR or
80 Z39.50). For the remainer of the connection's lifetime, you don't have
81 to worry about the underlying transport protocol at all - the <bf/COMSTACK/
82 will ensure that the correct mechanism is used.
84 We call the combined interfaces to <bf/ODR/, <bf/ASN/, and <bf/COMSTACK/ the service
85 level API. It's the API that most closely models the Z39.50/SR
86 service/protocol definition, and it provides unlimited access to all
87 fields and facilities of the protocol definitions.
89 The reason that the <bf/YAZ/ service-level API is a conglomerate of the
90 APIs from three different submodules is twofold. First, we wanted to allow the
91 user a choice of different options for each major task. For instance,
92 if you don't like the protocol API provided by <bf/ODR//<bf/ASN/, you
93 can use SNACC or BERUtils instead, and still have the benefits of the
94 transparent transport approach of the <bf/COMSTACK/ module. Secondly,
95 we realise that you may have to fit the toolkit into an existing
96 event-processing structure, in a way that
97 is incompatible with the <bf/COMSTACK/ interface or some other part of <bf/YAZ/.
99 <sect>Compilation and Installation
102 The latest version of the software will generally be found at
105 http://ftp.indexdata.dk/pub/yaz/
108 We have tried our best to keep the software portable, and on many
109 platforms, you should be able to compile everything with little or no changes.
110 So far, the software has been ported
111 to the following platforms with little or no difficulties.
121 <item>Data General DG/UX (with some CFLAGS tinkering)
125 <item>Non-unix systems
127 <item>Apple Macintosh (using the Codewarrior programming environment and the
128 GUSI socket libraries)
129 <item>MS Windows 95/NT (Win32)
134 If you move the software to other platforms, we'd be grateful if you'd
135 let us know about it. If you run into difficulties, we will try to help if we
136 can, and if you solve the problems, we would be happy to
137 include your fixes in the next release. So far, we have mostly avoided
138 #ifdefs for individual platforms, and we'd like to keep it that
139 way as far as it makes sense.
141 We maintain a mailing-list for the purpose of announcing new releases and
142 bug-fixes, as well as general discussion. Subscribe by sending mail to
143 <tt/yaz-request@indexdata.dk/. General questions and problems can be
144 directed at <tt/yaz-help@indexdata.dk/, or the address given at the top
150 Note that if your system doesn't have a native ANSI C compiler, you may
151 have to acquire one separately. We recommend gcc.
153 For UNIX we use GNU configure to create Makefiles for YAZ.
154 Generally it should be sufficient to run configure without options:
160 The configure script attempts to use use the C compiler specified by
161 the <tt/CC/ environment variable. If not set, GNU C will be used
162 if it is available. The <tt/CFLAGS/ environment variable holds options
163 to be passed to the C compiler. If you're using Bourne-compatible shell
164 you may pass something like this to use a particular C compiler with
165 optimization enabled:
167 CC=/opt/ccs/bin/cc CFLAGS=-O ./configure
170 To customize <bf/YAZ/ the configure script also accepts a set of options.
171 The most important are:
174 <tag><tt>-</tt><tt>-prefix </tt>path</tag> Specifies installation prefix. This is
175 only needed if you run <tt>make install</tt> later to perform a
176 "system" installation. The prefix is <tt>/usr/local</tt> if not
179 <tag><tt>-</tt><tt>-enable-comp </tt></tag> YAZ will be built using
180 the ASN.1 compiler for YAZ (default). If you wish to use the
181 old decoders (in sub directory asn) use <tt>--disable-comp</tt> instead.
183 <tag><tt>-</tt><tt>-enable-threads</tt></tag> YAZ will be built using
184 POSIX threads. Specifically, <tt>_REENTRANT</tt> will be defined
188 When configured, build the software by typing:
193 Developers that do modifications to <bf/YAZ/ may wish to run
194 <tt>make depend</tt> as well to generate object-header file
197 The following files are generated by make
199 <tag><tt>lib/libyaz.a</tt></tag> The <bf/YAZ/ programmers' library.
201 <tag><tt>ztest/yaz-ztest</tt></tag> A test Z39.50 server.
203 <tag><tt>client/yaz-client</tt></tag> A command mode Z39.50 client.
205 <tag><tt>yaz-config</tt></tag> A Bourne-shell script that holds build
206 settings for <bf/YAZ/.
210 If you wish to install <bf/YAZ/ in system directories such as /usr/local/bin,
211 /usr/local/lib) you can type:
217 You probably need to have root access in order to perform this.
218 You must specify the <tt>--prefix</tt> option for configure if you
219 going to install in anything but /usr/local/.
224 <bf/YAZ/ is shipped with "makefiles" for the NMAKE tool that comes
227 Start an MS-DOS prompt and switch the sub directory <tt>WIN</tt> where
228 the file <tt>makefile</tt> is located. Customize the installation
229 by editing the <tt>makefile</tt> file (for example by using notepad).
231 The following summarises the most important settings in that file:
234 <tag><tt>NEW_Z3950</tt></tag> If 1, the auto-generated decoder/encoders
235 for Z39.50 as written by the ASN.1 compiler will be used. If 0, the old
236 decoders for Z39.50 will be used. Note, when 1, the setting TCL should
237 point to the Tcl shell on your system.
238 <tag><tt>DEBUG</tt></tag> If set to 1, the software is
239 compiled with debugging libraries. If set to 0, the software
240 is compiled with release (non-debugging) libraries.
243 When satisfied with the settings in the makefile type
248 The following is generated upon successful compilation:
250 <tag><tt>bin/yaz.dll</tt></tag> A multithreaded DLL with everything
251 except the frontend server library.
252 <tag><tt>lib/yaz.lib</tt></tag> An import library for <tt>yaz.dll</tt>.
253 <tag><tt>lib/server.lib</tt></tag> The frontend server library.
254 <tag><tt>bin/yaz-ztest.exe</tt></tag> A test Z39.50 server.
255 <tag><tt>bin/yaz-client.exe</tt></tag> A command mode Z39.50 client.
263 The <bf/ASN/ module provides you with a set of C struct definitions for the
264 various PDUs of the protocol, as well as for the complex types
265 appearing within the PDUs. For the primitive data types, the C
266 representation often takes the form of an ordinary C language type,
267 such as <tt/int/. For ASN.1 constructs that have no direct
268 representation in C, such as general octet strings and bit strings,
269 the <bf/ODR/ module (see section <ref id="odr" name="ODR">) provides auxiliary
272 <sect1>Preparing PDUs
275 A structure representing a complex ASN.1 type doesn't in itself contain the
276 members of that type. Instead, the structure contains <it/pointers/ to
277 the members of the type. This is necessary, in part, to allow a mechanism for
278 specifying which of the optional structure (SEQUENCE) members are
279 present, and which are not. It follows that you will need to somehow
280 provide space for the individual members of the structure, and
281 set the pointers to refer to the members.
283 The conversion routines don't care how you allocate and maintain your
284 C structures - they just follow the pointers that you provide.
285 Depending on the complexity of your application, and your personal
286 taste, there are at least three different approaches that you may take
287 when you allocate the structures.
291 You can use static or automatic local variables in the function that
292 prepares the PDU. This is a simple approach, and it provides the most
293 efficient form of memory management. While it works well for flat
294 PDUs like the InitReqest, it will generally not be sufficient for say,
295 the generation of an arbitrarily complex RPN query structure.
297 You can individually create the structure and its members using the
298 <tt/malloc/(2) function. If you want to ensure that the data is freed when
299 it is no longer needed, you will have to define a function that
300 individually releases each member of a structure before freeing the
303 You can use the <tt/odr_malloc()/ function (see section <ref
305 name="Using ODR"> for details). When you use <tt/odr_malloc()/, you can release
307 allocated data in a single operation, independent of any pointers and
308 relations between the data. <tt/odr_malloc()/ is based on a
309 &dquot;nibble-memory&dquot;
310 scheme, in which large portions of memory are allocated, and then
311 gradually handed out with each call to <tt/odr_malloc()/. The next time you
312 call <tt/odr_reset()/, all of the memory allocated since the last call is
313 recycled for future use (actually, it is placed on a free-list).
315 You can combine all of the methods described here. This will often be
316 the most practical approach. For instance, you might use <tt/odr_malloc()/ to
317 allocate an entire structure and some of its elements, while you leave
318 other elements pointing to global or per-session default variables.
321 The <bf/ASN/ module provides an important aid in creating new PDUs. For
322 each of the PDU types (say, <tt/Z_InitRequest/), a function is provided
323 that allocates and initializes an instance of that PDU type for you.
324 In the case of the InitRequest, the function is simply named
325 <tt/zget_InitRequest()/, and it sets up reasonable default value for all of
326 the mandatory members. The optional members are generally initialized to null
327 pointers. This last aspect is very important: it ensures that if the
328 PDU definitions are extended after you finish your implementation
330 new versions of the protocol, say), you won't get into trouble with
331 uninitialized pointers in your structures. The functions use
333 allocate the PDUs and its members, so you can free everything again with a
334 single call to <tt/odr_reset()/. We strongly recommend that you use the
336 functions whenever you are preparing a PDU (in a C++ API, the
338 functions would probably be promoted to constructors for the
341 The prototype for the individual PDU types generally look like this:
344 Z_<type> *zget_<type>(ODR o);
350 Z_InitRequest *zget_InitRequest(ODR o);
353 The <bf/ODR/ handle should generally be your encoding stream, but it
356 As well as the individual PDU functions, a function <tt/zget_APDU()/ is
357 provided, which allocates a toplevel Z-APDU of the type requested:
360 Z_APDU *zget_APDU(ODR o, int which);
363 The <tt/which/ parameter is (of course) the discriminator belonging to the
364 <tt/Z_APDU/ CHOICE type. All of the interface described here is provided by
365 the <bf/ASN/ module, and you access it through the <tt/proto.h/ header file.
367 <sect1>Object Identifiers<label id="oid">
370 When you refer to object identifiers in your application, you need to
371 be aware that SR and Z39.50 use two different set of OIDs to refer to
372 the same objects. To handle this easily, <bf/YAZ/ provides a utility module
373 to <bf/ASN/ which provides an internal representation of the OIDs used in
374 both protocols. Each oid is described by a structure:
377 typedef struct oident
379 enum oid_proto proto;
380 enum oid_class class;
381 enum oid_value value;
382 int oidsuffix[OID_SIZE];
387 The <tt/proto/ field can be set to either <tt/PROTO_SR/ or
388 <tt/PROTO_Z3950/. The <tt/class/ might be, say, <tt/CLASS_RECSYN/, and the
389 <tt/value/ might be <tt/VAL_USMARC/ for the USMARC record format. Functions
392 int *oid_ent_to_oid(struct oident *ent, int *dst);
393 struct oident *oid_getentbyoid(int *o);
396 are provided to map between object identifiers and database entries.
397 If you store a member of the <tt/oid_proto/ type in your association
398 state information, it's a simple matter, at runtime, to generate the
399 correct OID when you need it. For decoding, you can simply ignore the
400 proto field, or if you're strict, you can verify that your peer is
401 using the OID family from the correct protocol. The <tt/desc/ field is
402 a short, human-readable name for the PDU, useful mainly for diagnostic
406 NOTE: The old function oid_getoidbyent still exists but is
407 not thread safe. Use oid_ent_to_oid instead and pass an array of
412 NOTE: Plans are underway to merge the two protocols into a single
413 definition, with one set of object identifiers. When this happens, the
414 oid module will no longer be required to support protocol
415 independence, but it should still be useful as a simple OID database.
421 In order to achieve extensibility and adaptability to different
422 application domains, the new version of the protocol defines many
423 structures outside of the main ASN.1 specification, referencing them
424 through ASN.1 EXTERNAL constructs. To simplify the construction and access
425 to the externally referenced data, the <bf/ASN/ module defines a
426 specialized version of the EXTERNAL construct, called <tt/Z_External/.
430 typedef struct Z_External
432 Odr_oid *direct_reference;
433 int *indirect_reference;
438 Z_External_single = 0,
440 Z_External_arbitrary,
444 Z_External_explainRecord,
445 Z_External_resourceReport1,
446 Z_External_resourceReport2
454 Odr_any *single_ASN1_type;
455 Odr_oct *octet_aligned;
456 Odr_bitmask *arbitrary;
460 Z_ExplainRecord *explainRecord;
461 Z_ResourceReport1 *resourceReport1;
462 Z_ResourceReport2 *resourceReport2;
470 When decoding, the <bf/ASN/ module will attempt to determine which
471 syntax describes the data by looking at the reference fields
472 (currently only the direct-reference). For ASN.1 structured data, you
473 need only consult the <tt/which/ field to determine the type of data.
474 You can the access the data directly through the union. When
475 constructing data for encoding, you set the union pointer to point to
476 the data, and set the <tt/which/ field accordingly. Remember also to
477 set the direct (or indirect) reference to the correct OID for the data
478 type. For non-ASN.1 data such as MARC records, use the
479 <tt/octet_aligned/ arm of the union.
481 Some servers return ASN.1 structured data values (eg. database
482 records) as BER-encoded records placed in the <tt/octet-aligned/
483 branch of the EXTERNAL CHOICE. The ASN-module will <it/not/
484 automatically decode these records. To help you decode the records in
485 the application, the function
488 Z_ext_typeent *z_ext_gettypebyref(oid_value ref);
491 Can be used to retrieve information about the known, external data
492 types. The function return a pointer to a static area, or NULL, if no
493 match for the given direct reference is found. The <tt/Z_ext_typeent/
497 typedef struct Z_ext_typeent
499 oid_value dref; /* the direct-reference OID value. */
500 int what; /* discriminator value for the external CHOICE */
501 Odr_fun fun; /* decoder function */
505 The <tt/what/ member contains the Z_External union discriminator value
506 for the given type: For the SUTRS record syntax, the value would be
507 <tt/Z_External_sutrs/. The <tt/fun/ member contains a pointer to the
508 function which encodes/decodes the given type. Again, for the SUTRS
509 record syntax, the value of <tt/fun/ would be <tt/z_SUTRS/ (a function
512 If you receive an EXTERNAL which contains an octet-string value that
513 you suspect of being an ASN.1-structured data value, you can use
514 <tt/z_ext_gettypebyref/ to look for the provided direct-reference. If
515 the return value is different from NULL, you can use the provided
516 function to decode the BER string (see section <ref id="odr-use"
519 If you want to <it/send/ EXTERNALs containing ASN.1-structured values
520 in the occtet-aligned branch of the CHOICE, this is possible too.
521 However, on the encoding phase, it requires a somewhat involved
522 juggling around of the various buffers involved.
524 If you need to add new, externally defined data types, you must update
525 the struct above, in the source file <tt/prt-ext.h/, as well as the
526 encoder/decoder in the file <tt/prt-ext.c/. When changing the latter,
527 remember to update both the <tt/arm/ arrary and the list <tt/type_table/,
528 which drives the CHOICE biasing that is necessary to tell the
529 different, structured types apart on decoding.
532 NOTE: Eventually, the EXTERNAL processing will most likely
533 automatically insert the correct OIDs or indirect-refs. First,
534 however, we need to determine how application-context management
535 (specifically the presentation-context-list) should fit into the
539 <sect1>PDU Contents Table
542 We include, for reference, a listing of the fields of each top-level
543 PDU, as well as their default settings.
548 Field Type Default value
550 referenceId Z_ReferenceId NULL
551 protocolVersion Odr_bitmask Empty bitmask
552 options Odr_bitmask Empty bitmask
553 preferredMessageSize int 30*1024
554 maximumRecordSize int 30*1024
555 idAuthentication Z_IdAuthentication NULL
556 implementationId char* "YAZ (id=81)"
557 implementationName char* "Index Data/YAZ"
558 implementationVersion char* YAZ_VERSION
559 userInformationField Z_UserInformation NULL
560 otherInfo Z_OtherInformation NULL
564 Field Type Default value
566 referenceId Z_ReferenceId NULL
567 protocolVersion Odr_bitmask Empty bitmask
568 options Odr_bitmask Empty bitmask
569 preferredMessageSize int 30*1024
570 maximumRecordSize int 30*1024
572 implementationId char* "YAZ (id=81)"
573 implementationName char* "Index Data/YAZ"
574 implementationVersion char* YAZ_VERSION
575 userInformationField Z_UserInformat.. NULL
576 otherInfo Z_OtherInformation NULL
580 Field Type Default value
582 referenceId Z_ReferenceId NULL
583 smallSetUpperBound int 0
584 largeSetLowerBound int 1
585 mediumSetPresentNumber int 0
586 replaceIndicator bool_t TRUE
587 resultSetName char* "default"
588 num_databaseNames int 0
589 databaseNames char** NULL
590 smallSetElementSetNames Z_ElementSetNames NULL
591 mediumSetElementSetNames Z_ElementSetNames NULL
592 preferredRecordSyntax Odr_oid NULL
594 additionalSearchInfo Z_OtherInformation NULL
595 otherInfo Z_OtherInformation NULL
599 Field Type Default value
601 referenceId Z_ReferenceId NULL
603 numberOfRecordsReturned int 0
604 nextResultSetPosition int 0
605 searchStatus bool_t TRUE
606 resultSetStatus int NULL
607 presentStatus int NULL
608 records Z_Records NULL
609 additionalSearchInfo Z_OtherInformation NULL
610 otherInfo Z_OtherInformation NULL
614 Field Type Default value
616 referenceId Z_ReferenceId NULL
617 resultSetId char* "default"
618 resultSetStartPoint int 1
619 numberOfRecordsRequested int 10
621 additionalRanges Z_Range NULL
622 recordComposition Z_RecordComposition NULL
623 preferredRecordSyntax Odr_oid NULL
624 maxSegmentCount int NULL
625 maxRecordSize int NULL
626 maxSegmentSize int NULL
627 otherInfo Z_OtherInformation NULL
631 Field Type Default value
633 referenceId Z_ReferenceId NULL
634 numberOfRecordsReturned int 0
635 nextResultSetPosition int 0
636 presentStatus int Z_PRES_SUCCESS
637 records Z_Records NULL
638 otherInfo Z_OtherInformation NULL
640 Z_DeleteResultSetRequest
641 ------------------------
642 Field Type Default value
644 referenceId Z_ReferenceId NULL
645 deleteFunction int Z_DeleteRequest_list
647 resultSetList char** NULL
648 otherInfo Z_OtherInformation NULL
650 Z_DeleteResultSetResponse
651 -------------------------
652 Field Type Default value
654 referenceId Z_ReferenceId NULL
655 deleteOperationStatus int Z_DeleteStatus_success
657 deleteListStatuses Z_ListStatus** NULL
658 numberNotDeleted int NULL
659 num_bulkStatuses int 0
660 bulkStatuses Z_ListStatus NULL
661 deleteMessage char* NULL
662 otherInfo Z_OtherInformation NULL
666 Field Type Default value
668 referenceId Z_ReferenceId NULL
669 num_databaseNames int 0
670 databaseNames char** NULL
671 attributeSet Odr_oid NULL
672 termListAndStartPoint Z_AttributesPlus... NULL
674 numberOfTermsRequested int 20
675 preferredPositionInResponse int NULL
676 otherInfo Z_OtherInformation NULL
680 Field Type Default value
682 referenceId Z_ReferenceId NULL
684 scanStatus int Z_Scan_success
685 numberOfEntriesReturned int 0
686 positionOfTerm int NULL
687 entries Z_ListEntris NULL
688 attributeSet Odr_oid NULL
689 otherInfo Z_OtherInformation NULL
691 Z_TriggerResourceControlRequest
692 -------------------------------
693 Field Type Default value
695 referenceId Z_ReferenceId NULL
696 requestedAction int Z_TriggerResourceCtrl_resou..
697 prefResourceReportFormat Odr_oid NULL
698 resultSetWanted bool_t NULL
699 otherInfo Z_OtherInformation NULL
701 Z_ResourceControlRequest
702 ------------------------
703 Field Type Default value
705 referenceId Z_ReferenceId NULL
706 suspendedFlag bool_t NULL
707 resourceReport Z_External NULL
708 partialResultsAvailable int NULL
709 responseRequired bool_t FALSE
710 triggeredRequestFlag bool_t NULL
711 otherInfo Z_OtherInformation NULL
713 Z_ResourceControlResponse
714 -------------------------
715 Field Type Default value
717 referenceId Z_ReferenceId NULL
718 continueFlag bool_t TRUE
719 resultSetWanted bool_t NULL
720 otherInfo Z_OtherInformation NULL
722 Z_AccessControlRequest
723 ----------------------
724 Field Type Default value
726 referenceId Z_ReferenceId NULL
727 which enum Z_AccessRequest_simpleForm;
729 otherInfo Z_OtherInformation NULL
731 Z_AccessControlResponse
732 -----------------------
733 Field Type Default value
735 referenceId Z_ReferenceId NULL
736 which enum Z_AccessResponse_simpleForm
738 diagnostic Z_DiagRec NULL
739 otherInfo Z_OtherInformation NULL
743 Field Type Default value
745 referenceId Z_ReferenceId NULL
746 numberOfRecordsReturned int value=0
747 num_segmentRecords int 0
748 segmentRecords Z_NamePlusRecord NULL
749 otherInfo Z_OtherInformation NULL
753 Field Type Default value
755 referenceId Z_ReferenceId NULL
756 closeReason int Z_Close_finished
757 diagnosticInformation char* NULL
758 resourceReportFormat Odr_oid NULL
759 resourceFormat Z_External NULL
760 otherInfo Z_OtherInformation NULL
763 <sect>Supporting Tools
766 In support of the service API - primarily the ASN module, which
767 provides the programmatic interface to the Z39.50 APDUs, YAZ contains
768 a collection of tools that support the development of applications.
770 <sect1>Query Syntax Parsers
773 Since the type-1 (RPN) query structure has no direct, useful string
774 representation, every origin application needs to provide some form of
775 mapping from a local query notation or representation to a
776 <tt/Z_RPNQuery/ structure. Some programmers will prefer to construct
777 the query manually, perhaps using <tt/odr_malloc()/ to simplify memory
778 management. The YAZ distribution includes two separate,
779 query-generating tools that may be of use to you.
781 <sect2>Prefix Query Format
784 Since RPN or reverse polish notation is really just a fancy way of
785 describing a suffix notation format (operator follows operands), it
786 would seem that the confusion is total when we now introduce a prefix
787 notation for RPN. The reason is one of simple laziness - it's somewhat
788 simpler to interpret a prefix format, and this utility was designed
789 for maximum simplicity, to provide a baseline representation for use
790 in simple test applications and scripting environments (like Tcl). The
791 demonstration client included with YAZ uses the PQF.
793 The PQF is defined by the pquery module in the YAZ library. The
794 <tt/pquery.h/ file provides the declaration of the functions
797 Z_RPNQuery *p_query_rpn (ODR o, oid_proto proto, const char *qbuf);
799 Z_AttributesPlusTerm *p_query_scan (ODR o, oid_proto proto,
800 Odr_oid **attributeSetP, const char *qbuf);
802 int p_query_attset (const char *arg);
805 The function <tt/p_query_rpn()/ takes as arguments an <bf/ODR/ stream
806 (see section <ref id="odr" name="The ODR Module">) to provide a memory
807 source (the structure created is released on the next call to
808 <tt/odr_reset()/ on the stream/), a protocol identifier (one of the
809 constants <tt/PROTO_Z3950/ and <tt/PROTO_SR/), an attribute set
810 reference, and finally a null-terminated string holding the query
813 If the parse went well, <tt/p_query_rpn()/ returns a pointer to a
814 <tt/Z_RPNQuery/ structure which can be placed directly into a
815 <tt/Z_SearchRequest/.
817 The <tt/p_query_attset/ specifies which attribute set to use if
818 the query doesn't specify one by the <tt/@attrset/ operator. The
819 <tt/p_query_attset/ returns 0 if the argument is a valid attribute
820 set specifier; otherwise the function returns -1.
822 The grammar of the PQF is as follows:
825 Query ::= [ AttSet ] QueryStruct.
829 QueryStruct ::= { Attribute } Simple | Complex.
831 Attribute ::= '@attr' AttributeType '=' AttributeValue.
833 AttributeType ::= integer.
835 AttributeValue ::= integer.
837 Complex ::= Operator QueryStruct QueryStruct.
839 Operator ::= '@and' | '@or' | '@not' | '@prox' Proximity.
841 Simple ::= ResultSet | Term.
843 ResultSet ::= '@set' string.
845 Term ::= string | '"' string '"'.
847 Proximity ::= Exclusion Distance Ordered Relation WhichCode UnitCode.
849 Exclusion ::= '1' | '0' | 'void'.
851 Distance ::= integer.
853 Ordered ::= '1' | '0'.
855 Relation ::= integer.
857 WhichCode ::= 'known' | 'private' | integer.
859 UnitCode ::= integer.
862 You will note that the syntax above is a fairly faithful
863 representation of RPN, except for the <tt/Attibute/, which has been
864 moved a step away from the term, allowing you to associate one or more
865 attributes with an entire query structure. The parser will
866 automatically apply the given attributes to each term as required.
868 The following are all examples of valid queries in the PQF.
875 @or "dylan" "zimmerman"
879 @or @and bob dylan @set Result-1
881 @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
883 @attr 4=1 @attr 1=4 "self portrait"
885 @prox 0 3 1 2 k 2 dylan zimmerman
888 <sect2>Common Command Language
891 Not all users enjoy typing in prefix query structures and numerical
892 attribute values, even in a minimalistic test client. In the library
893 world, the more intuitive Common Command Language (or ISO 8777) has
894 enjoyed some popularity - especially before the widespread
895 availability of graphical interfaces. It is still useful in
896 applications where you for some reason or other need to provide a
897 symbolic language for expressing boolean query structures.
899 The EUROPAGATE research project working under the Libraries programme
900 of the European Commission's DG XIII has, amongst other useful tools,
901 implemented a general-purpose CCL parser which produces an output
902 structure that can be trivially converted to the internal RPN
903 representation of YAZ (The <tt/Z_RPNQuery/ structure). Since the CCL
904 utility - along with the rest of the software produced by EUROPAGATE -
905 is made freely available on a liberal license, it is included as a
910 The CCL parser obeys the following grammar for the FIND argument.
911 The syntax is annotated by in the lines prefixed by <tt/--/.
914 CCL-Find ::= CCL-Find Op Elements
917 Op ::= "and" | "or" | "not"
918 -- The above means that Elements are separated by boolean operators.
920 Elements ::= '(' CCL-Find ')'
923 | Qualifiers Relation Terms
924 | Qualifiers Relation '(' CCL-Find ')'
925 | Qualifiers '=' string '-' string
926 -- Elements is either a recursive definition, a result set reference, a
927 -- list of terms, qualifiers followed by terms, qualifiers followed
928 -- by a recursive definition or qualifiers in a range (lower - upper).
930 Set ::= 'set' = string
931 -- Reference to a result set
933 Terms ::= Terms Prox Term
935 -- Proximity of terms.
939 -- This basically means that a term may include a blank
941 Qualifiers ::= Qualifiers ',' string
943 -- Qualifiers is a list of strings separated by comma
945 Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
946 -- Relational operators. This really doesn't follow the ISO8777
950 -- Proximity operator
954 The following queries are all valid:
965 (dylan and bob) or set=1
968 Assuming that the qualifiers <tt/ti/, <tt/au/ and <tt/date/
969 are defined we may use:
973 au=(bob dylan and slow train coming)
975 date>1980 and (ti=((self portrait)))
978 <sect3>CCL Qualifiers
981 Qualifiers are used to direct the search to a particular searchable
982 index, such as title (ti) and author indexes (au). The CCL standard
983 itself doesn't specify a particular set of qualifiers, but it does
984 suggest a few short-hand notations. You can customize the CCL parser
985 to support a particular set of qualifiers to relect the current target
986 profile. Traditionally, a qualifier would map to a particular
987 use-attribute within the BIB-1 attribute set. However, you could also
988 define qualifiers that would set, for example, the
991 Consider a scenario where the target support ranked searches in the
992 title-index. In this case, the user could specify
994 ti,ranked=knuth computer
996 and the <tt/ranked/ would map to structure=free-form-text
997 (4=105) and the <tt/ti/ would map to title (1=4).
999 A "profile" with a set predefined CCL qualifiers can be read from a
1000 file. The YAZ client reads its CCL qualifiers from a file named
1001 <tt/default.bib/. Each line in the file has the form:
1003 <it/qualifier-name/ <it/type/=<it/val/ <it/type/=<it/val/ ...
1005 where <it/qualifier-name/ is the name of the qualifier to be used
1006 (eg. <tt/ti/), <it/type/ is a BIB-1 category type and <it/val/ is the
1007 corresponding BIB-1 attribute value. The <it/type/ can be either
1008 numeric or it may be either <tt/u/ (use), <tt/r/ (relation), <tt/p/
1009 (position), <tt/s/ (structure), <tt/t/ (truncation) or <tt/c/
1010 (completeness). The <it/qualifier-name/ <tt/term/ has a special
1011 meaning. The types and values for this definition is used when <it/no/
1012 qualifier is present.
1014 Consider the following definition:
1020 Two qualifiers are defined, <tt/ti/ and <tt/au/. They both set the
1021 structure-attribute to phrase (1). <tt/ti/ sets the use-attribute to
1022 4. <tt/au/ sets the use-attribute to 1. When no qualifiers are used
1023 in the query the structure-attribute is set to free-form-text (105).
1028 All public definitions can be found in the header file <tt/ccl.h/.
1029 A profile identifier is of type <tt/CCL_bibset/. A profile must be
1030 created with the call to the function <tt/ccl_qual_mk/ which returns
1031 a profile handle of type <tt/CCL_bibset/.
1033 To read a file containing qualifier definitions the function
1034 <tt/ccl_qual_file/ may be convenient. This function takes an already
1035 opened <tt/FILE/ handle pointer as argument along with a
1036 <tt/CCL_bibset/ handle.
1038 To parse a simple string with a FIND query use the function
1040 struct ccl_rpn_node *ccl_find_str (CCL_bibset bibset, const char *str,
1041 int *error, int *pos);
1043 which takes the CCL profile (<tt/bibset/) and query (<tt/str/) as
1044 input. Upon successful completion the RPN tree is returned. If an
1045 error eccur, such as a syntax error, the integer pointed to by
1046 <tt/error/ holds the error code and <tt/pos/ holds the offset inside
1047 query string in which the parsing failed.
1049 An english representation of the error may be obtained by calling
1050 the <tt/ccl_err_msg/ function. The error codes are listed in
1053 To convert the CCL RPN tree (type <tt/struct ccl_rpn_node */) to the
1054 Z_RPNQuery of YAZ the function <tt/ccl_rpn_query/ must be used. This
1055 function which is part of YAZ is implemented in <tt/yaz-ccl.c/.
1056 After calling this function the CCL RPN tree is probably no longer
1057 needed. The <tt/ccl_rpn_delete/ destroys the CCL RPN tree.
1059 A CCL profile may be destroyed by calling the <tt/ccl_qual_rm/
1062 The token names for the CCL operators may be changed by setting the
1063 globals (all type <tt/char */) <tt/ccl_token_and/, <tt/ccl_token_or/,
1064 <tt/ccl_token_not/ and <tt/ccl_token_set/.
1065 An operator may have aliases, i.e. there may be more than one name for
1066 the operator. To do this, separate each alias with a space character.
1068 <sect1>Object Identifiers
1071 The basic YAZ representation of an OID is an array of integers,
1072 terminated with the value -1. The <bf/ODR/ module provides two
1073 utility-functions to create and copy this type of data elements:
1076 Odr_oid *odr_getoidbystr(ODR o, char *str);
1079 Creates an OID based on a string-based representation using dots (.)
1080 to separate elements in the OID.
1083 Odr_oid *odr_oiddup(ODR odr, Odr_oid *o);
1086 Creates a copy of the OID referenced by the <it/o/ parameter. Both
1087 functions take an <bf/ODR/ stream as parameter. This stream is used to
1088 allocate memory for the data elements, which is released on a
1089 subsequent call to <tt/odr_reset()/ on that stream.
1091 The <bf/OID/ module provides a higher-level representation of the
1092 family of object identifers which describe the Z39.50 protocol and its
1093 related objects. The definition of the module interface is given in
1094 the <tt/oid.h/ file.
1096 The interface is mainly based on the <tt/oident/ structure. The
1097 definition of this structure looks like this:
1100 typedef struct oident
1105 int oidsuffix[OID_SIZE];
1110 The <it/proto/ field takes one of the values
1117 If you don't care about talking to SR-based implementations (few
1118 exist, and they may become fewer still if and when the ISO SR and ANSI
1119 Z39.50 documents are merged into a single standard), you can ignore
1120 this field on incoming packages, and always set it to PROTO_Z3950
1121 for outgoing packages.
1123 The <it/oclass/ field takes one of the values
1143 corresponding to the OID classes defined by the Z39.50 standard.
1145 Finally, the <it/value/ field takes one of the values
1201 again, corresponding to the specific OIDs defined by the standard.
1203 The <it/desc/ field contains a brief, mnemonic name for the OID in
1209 struct oident *oid_getentbyoid(int *o);
1212 takes as argument an OID, and returns a pointer to a static area
1213 containing an <tt/oident/ structure. You typically use this function
1214 when you receive a PDU containing an OID, and you wish to branch out
1215 depending on the specific OID value.
1220 int *oid_ent_to_oid(struct oident *ent, int *dst);
1223 Takes as argument an <tt/oident/ structure - in which the <it/proto/,
1224 <it/oclass/, and <it/value/ fields are assumed to be set correctly -
1225 and returns a pointer to a the buffer as given by <it/dst/
1227 representation of the corresponding OID. The function returns
1228 NULL and the array dst is unchanged if a mapping couldn't place.
1229 The array <it/dst/ should be at least of size <tt>OID_SIZE</tt>.
1231 The <tt/oid_ent_to_oid()/ function can be used whenever you need to
1232 prepare a PDU containing one or more OIDs. The separation of the
1233 <it/protocol/ element from the remainer of the OID-description makes
1234 it simple to write applications that can communicate with either
1235 Z39.50 or OSI SR-based applications.
1240 oid_value oid_getvalbyname(const char *name);
1243 takes as argument a mnemonic OID name, and returns the <it/value/
1244 field of the first entry in the database that contains the given name
1245 in its <it/desc/ field.
1247 Finally, the module provides the following utility functions, whose
1248 meaning should be obvious:
1251 void oid_oidcpy(int *t, int *s);
1252 void oid_oidcat(int *t, int *s);
1253 int oid_oidcmp(int *o1, int *o2);
1254 int oid_oidlen(int *o);
1258 NOTE: The <bf/OID/ module has been criticized - and perhaps rightly so
1259 - for needlessly abstracting the
1260 representation of OIDs. Other toolkits use a simple
1261 string-representation of OIDs with good results. In practice, we have
1262 found the interface comfortable and quick to work with, and it is a
1263 simple matter (for what it's worth) to create applications compatible
1264 with both ISO SR and Z39.50. Finally, the use of the <tt/oident/
1265 database is by no means mandatory. You can easily create your
1266 own system for representing OIDs, as long as it is compatible with the
1267 low-level integer-array representation of the ODR module.
1270 <sect1>Nibble Memory
1273 Sometimes when you need to allocate and construct a large,
1274 interconnected complex of structures, it can be a bit of a pain to
1275 release the associated memory again. For the structures describing the
1276 Z39.50 PDUs and related structures, it is convenient to use the
1277 memory-management system of the <bf/ODR/ subsystem (see
1278 <ref id="odr-use" name="Using ODR">). However, in some circumstances
1279 where you might otherwise benefit from using a simple nibble memory
1280 management system, it may be impractical to use <tt/odr_malloc()/ and
1281 <bf/odr_reset()/. For this purpose, the memory manager which also
1282 supports the <bf/ODR/ streams is made available in the <bf/NMEM/
1283 module. The external interface to this module is given in the
1286 The following prototypes are given:
1289 NMEM nmem_create(void);
1290 void nmem_destroy(NMEM n);
1291 void *nmem_malloc(NMEM n, int size);
1292 void nmem_reset(NMEM n);
1293 int nmem_total(NMEM n);
1294 void nmem_init(void);
1297 The <tt/nmem_create()/ function returns a pointer to a memory control
1298 handle, which can be released again by <tt/nmem_destroy()/ when no
1299 longer needed. The function <tt/nmem_malloc()/ allocates a block of
1300 memory of the requested size. A call to <tt/nmem_reset()/ or
1301 <tt/nmem_destroy()/ will release all memory allocated on the handle
1302 since it was created (or since the last call to
1303 <tt/nmem_reset()/. The function <tt/nmem_total()/ returns the number
1304 of bytes currently allocated on the handle.
1306 Note that the nibble memory pool is shared amonst threads. Posix
1307 mutex'es and WIN32 Critical sections are introduced to keep the
1308 module thread safe. On WIN32 function <tt/nmem_init()/ initialises
1309 the Critical Section handle and should be called once before any
1310 other nmem function is used.
1312 <sect>The ODR Module<label id="odr">
1317 <bf/ODR/ is the BER-encoding/decoding subsystem of <bf/YAZ/. Care as been taken
1318 to isolate <bf/ODR/ from the rest of the package - specifically from the
1319 transport interface. <bf/ODR/ may be used in any context where basic
1320 ASN.1/BER representations are used.
1322 If you are only interested in writing a Z39.50 implementation based on
1323 the PDUs that are already provided with <bf/YAZ/, you only need to concern
1324 yourself with the section on managing ODR streams (section
1325 <ref id="odr-use" name="Using ODR">). Only if you need to
1326 implement ASN.1 beyond that which has been provided, should you
1327 worry about the second half of the documentation
1328 (section <ref id="odr-prog" name="Programming with ODR">). If you use
1329 one of the higher-level interfaces, you can skip this section entirely.
1331 This is important, so we'll repeat it for emphasis: <it>You do not
1332 need to read section <ref id="odr-prog" name="Programming with ODR"> to
1333 implement Z39.50 with <bf/YAZ/.</it>
1335 If you need a part of the protocol that isn't already in <bf/YAZ/, you
1337 contact the authors before going to work on it yourself: We
1338 might already be working on it. Conversely, if you implement a useful
1339 part of the protocol before us, we'd be happy to include it in a
1342 <sect1>Using ODR<label id="odr-use">
1348 Conceptually, the ODR stream is the source of encoded data in the
1349 decoding mode; when encoding, it is the receptacle for the encoded
1350 data. Before you can use an ODR stream it must be allocated. This is
1351 done with the function
1354 ODR odr_createmem(int direction);
1357 The <tt/odr_createmem()/ function takes as argument one of three manifest
1358 constants: <tt/ODR_ENCODE/, <tt/ODR_DECODE/, or <tt/ODR_PRINT/. An
1360 in only one mode - it is not possible to change its mode once it's
1361 selected. Typically, your program will allocate at least two ODR
1362 streams - one for decoding, and one for encoding.
1364 When you're done with the stream, you can use
1367 void odr_destroy(ODR o);
1370 to release the resources allocated for the stream.
1372 <sect2>Memory Management<label id="memory">
1375 Two forms of memory management take place in the ODR system. The first
1376 one, which has to do with allocating little bits of memory (sometimes
1377 quite large bits of memory, actually) when a protocol package is
1378 decoded, and turned into a complex of interlinked structures. This
1379 section deals with this system, and how you can use it for your own
1380 purposes. The next section deals with the memory management which is
1381 required when encoding data - to make sure that a large enough buffer is
1382 available to hold the fully encoded PDU.
1384 The <bf/ODR/ module has its own memory management system, which is
1385 used whenever memory is required. Specifically, it is used to allocate
1386 space for data when decoding incoming PDUs. You can use the memory
1387 system for your own purposes, by using the function
1390 void *odr_malloc(ODR o, int size);
1393 You can't use the normal <tt/free/(2) routine to free memory allocated
1394 by this function, and <bf/ODR/ doesn't provide a parallel function. Instead,
1398 void odr_reset(ODR o, int size);
1401 when you are done with the
1402 memory: Everything allocated since the last call to <tt/odr_reset()/ is
1403 released. The <tt/odr_reset()/ call is also required to clear up an
1404 error condition on a stream.
1409 int odr_total(ODR o);
1412 returns the number of bytes allocated on the stream since the last call to
1415 The memory subsystem of <bf/ODR/ is fairly efficient at allocating and
1416 releasing little bits of memory. Rather than managing the individual,
1417 small bits of space, the system maintains a freelist of larger chunks
1418 of memory, which are handed out in small bits. This scheme is
1419 generally known as a <it/nibble memory/ system. It is very useful for
1420 maintaing short-lived constructions such as protocol PDUs.
1422 If you want to retain a bit of memory beyond the next call to
1423 <tt/odr_reset()/, you can use the function
1426 ODR_MEM odr_extract_mem(ODR o);
1429 This function will give you control of the memory recently allocated
1430 on the ODR stream. The memory will live (past calls to <tt/odr_reset()/),
1431 until you call the function
1434 void odr_release_mem(ODR_MEM p);
1437 The opaque <tt/ODR_MEM/ handle has no other purpose than referencing
1438 the memory block for you until you want to release it.
1440 You can use <tt/odr_extract_mem()/ repeatedly between allocating data,
1441 to retain individual control of separate chunks of data.
1443 <sect2>Encoding and Decoding Data
1446 When encoding data, the ODR stream will write the encoded octet string
1447 in an internal buffer. To retrieve the data, use the function
1450 char *odr_getbuf(ODR o, int *len, int *size);
1453 The integer pointed to by len is set to the length of the encoded
1454 data, and a pointer to that data is returned. *<tt/size/ is set to the size
1455 of the buffer (unless <tt/size/ is null, signalling that you are
1456 not interested in the size). The next call to a primitive function
1457 using the same ODR stream will overwrite the data, unless a different
1458 buffer has been supplied using the call
1461 void odr_setbuf(ODR o, char *buf, int len, int can_grow);
1464 which sets the encoding (or decoding) buffer used by <tt/o/ to
1466 using the length <tt/len/. Before a call to an encoding function,
1467 you can use <tt/odr_setbuf()/ to provide the stream with an encoding buffer
1468 of sufficient size (length). The <tt/can_grow/ parameter tells the encoding
1469 ODR stream whether it is allowed to use <tt/realloc/(2) to increase the size
1470 of the buffer when necessary. The default condition of a new encoding
1471 stream is equivalent to the results of calling
1474 odr_setbuf(stream, 0, 0, 1);
1477 In this case, the stream will allocate and reallocate memory as
1478 necessary. The stream reallocates memory by repeatedly doubling the
1479 size of the buffer - the result is that the buffer will typically
1480 reach its maximum, working size with only a small number of reallocation
1481 operations. The memory is freed by the stream when the latter is destroyed,
1482 unless it was assigned by the user with the <tt/can_grow/
1483 parameter set to zero (in this case, you are expected to retain
1484 control of the memory yourself).
1486 To assume full control of an encoded buffer, you must first call
1487 <tt/odr_getbuf()/ to fetch the buffer and its length. Next, you should
1488 call <tt/odr_setbuf()/ to provide a different buffer (or a null
1489 pointer) to the stream. In the simplest case, you will reuse the same
1490 buffer over and over again, and you will just need to call
1491 <tt/odr_getbuf()/ after each encoding operation to get the length and
1492 address of the buffer. Note that the stream may reallocate the buffer
1493 during an encoding operation, so it is necessary to retrieve the
1494 correct address after each encoding operation.
1496 It is important to realise that the ODR stream will not release this
1497 memory when you call <tt/odr_reset()/: It will merely update its
1498 internal pointers to prepare for the encoding of a new data value.
1499 When the stream is released by the <tt/odr_destroy()/ function, the
1500 memory given to it by odr_setbuf will be released <it/only/ if the
1501 <tt/can_grow/ parameter to <tt/odr_setbuf()/ was nonzero. The
1502 <tt/can_grow/ parameter, in other words, is a way of signalling who
1503 is to own the buffer, you or the ODR stream. If you never call
1504 <tt/odr_setbuf()/ on your encoding stream, which is typically the
1505 case, the buffer allocated by the stream will belong to the stream by
1508 When you wish to decode data, you should first call <tt/odr_setbuf()/, to
1509 tell the decoding stream where to find the encoded data, and how long
1510 the buffer is (the <tt/can_grow/ parameter is ignored by a decoding
1511 stream). After this, you can call the function corresponding to the
1512 data you wish to decode (eg, <tt/odr_integer()/ odr <tt/z_APDU()/).
1514 Examples of encoding/decoding functions:
1517 int odr_integer(ODR o, int **p, int optional, const char *name);
1519 int z_APDU(ODR o, Z_APDU **p, int optional, const char *name);
1522 If the data is absent (or
1523 doesn't match the tag corresponding to the type), the return value
1524 will be either 0 or 1 depending on the <tt/optional/ flag. If
1526 is 0 and the data is absent, an error flag will be raised in the
1527 stream, and you'll need to call <tt/odr_reset()/ before you can use the
1528 stream again. If <tt/optional/ is nonzero, the pointer <it/pointed to/ by
1529 <tt/p/ will be set to the null value, and the function will return 1.
1530 The <tt/name/ argument is used to pretty-print the tag in question.
1531 It may be set to <tt/NULL/ if pretty-printing is not desired.
1533 If the data value is found where it's expected, the pointer
1534 <it/pointed to/ by the <tt/p/ argument will be set to point to the
1536 space for the type will be allocated and owned by the ODR stream, and
1537 it will live until you call <tt/odr_reset()/ on the stream. You cannot use
1538 <tt/free/(2) to release the memory. You can decode several data elements
1539 (by repeated calls to <tt/odr_setbuf()/ and your decoding function), and
1540 new memory will be allocated each time. When you do call
1542 everything decoded since the last call to <tt/odr_reset()/ will be
1545 The use of the double indirection can be a little confusing at first
1546 (its purpose will become clear later on, hopefully),
1547 so an example is in order. We'll encode an integer value, and
1548 immediately decode it again using a different stream. A useless, but
1549 informative operation.
1552 void do_nothing_useful(int value)
1555 int *valp, *resvalp;
1559 /* allocate streams */
1560 if (!(encode = odr_createmem(ODR_ENCODE)))
1562 if (!(decode = odr_createmem(ODR_DECODE)))
1566 if (odr_integer(encode, &ero;valp, 0, 0) == 0)
1568 printf("encoding went bad\n");
1571 bufferp = odr_getbuf(encode, &ero;len);
1572 printf("length of encoded data is %d\n", len);
1574 /* now let's decode the thing again */
1575 odr_setbuf(decode, bufferp, len);
1576 if (odr_integer(decode, &ero;resvalp, 0, 0) == 0)
1578 printf("decoding went bad\n");
1581 printf("the value is %d\n", *resvalp);
1584 odr_destroy(encode);
1585 odr_destroy(decode);
1589 This looks like a lot of work, offhand. In practice, the ODR streams
1590 will typically be allocated once, in the beginning of your program (or at the
1591 beginning of a new network session), and the encoding and decoding
1592 will only take place in a few, isolated places in your program, so the
1593 overhead is quite manageable.
1598 The encoding/decoding functions all return 0 when an error occurs.
1599 Until you call <tt/odr_reset()/, you cannot use the stream again, and
1600 any function called will immediately return 0.
1602 To provide information to the programmer or administrator, the
1606 void odr_perror(ODR o, char *message);
1609 is provided, which prints the <tt/message/ argument to <tt/stderr/
1610 along with an error message from the stream.
1612 You can also use the function
1615 int odr_geterror(ODR o);
1618 to get the current error number from the screen. The number will be
1619 one of these constants:
1622 <tag/OMEMORY/Memory allocation failed.
1623 <tag/OSYSERR/A system- or library call has failed. The standard
1624 diagnostic variable <tt/errno/
1625 should be examined to determine the actual error.
1626 <tag/OSPACE/No more space for encoding. This will only occur when the user has
1627 explicitly provided a buffer for an encoding stream without allowing
1628 the system to allocate more space.
1629 <tag/OREQUIRED/This is a common protocol error; A required data element was
1631 encoding or decoding.
1632 <tag/OUNEXPECTED/An unexpected data element was found during decoding.
1633 <tag/OOTHER/Other error. This is typically an indication of misuse of
1634 the <bf/ODR/ system by the programmer, and also that the diagnostic
1635 system isn't as good as it should be, yet.
1638 The character string array
1641 char *odr_errlist[]
1644 can be indexed by the error code to obtain a human-readable
1645 representation of the problem.
1647 <sect2>Summary and Synopsis
1653 ODR odr_createmem(int direction);
1655 void odr_destroy(ODR o);
1657 void odr_reset(ODR o);
1659 char *odr_getbuf(ODR o, int *len);
1661 void odr_setbuf(ODR o, char *buf, int len);
1663 void *odr_malloc(ODR o, int size);
1665 ODR_MEM odr_extract_mem(ODR o);
1667 void odr_release_mem(ODR_MEM r);
1669 int odr_geterror(ODR o);
1671 void odr_perror(char *message);
1673 extern char *odr_errlist[];
1676 <sect1>Programming with ODR<label id="odr-prog">
1679 The API of <bf/ODR/ is designed to reflect the structure of ASN.1, rather
1680 than BER itself. Future releases may be able to represent data in
1681 other external forms.
1683 The interface is based loosely on that of the Sun Microsystems XDR routines.
1684 Specifically, each function which corresponds to an ASN.1 primitive
1685 type has a dual function. Depending on the settings of the ODR
1686 stream which is supplied as a parameter, the function may be used
1687 either to encode or decode data. The functions that can be built
1688 using these primitive functions, to represent more complex datatypes, share
1689 this quality. The result is that you only have to enter the definition
1690 for a type once - and you have the functionality of encoding, decoding
1691 (and pretty-printing) all in one unit. The resulting C source code is
1692 quite compact, and is a pretty straightforward representation of the
1693 source ASN.1 specification. Although no ASN.1 compiler is supplied
1694 with <bf/ODR/ at this time, it shouldn't be too difficult to write one, or
1695 perhaps even to adapt an existing compiler to output <bf/ODR/ routines
1696 (not surprisingly, writing encoders/decoders using <bf/ODR/ turns out
1699 In many cases, the model of the XDR functions works quite well in this
1701 others, it is less elegant. Most of the hassle comes from the optional
1702 SEQUENCE memebers which don't exist in XDR.
1704 <sect2>The Primitive ASN.1 Types
1707 ASN.1 defines a number of primitive types (many of which correspond
1709 primitive types in structured programming languages, such as C).
1714 The <bf/ODR/ function for encoding or decoding (or printing) the ASN.1
1715 INTEGER type looks like this:
1718 int odr_integer(ODR o, int **p, int optional, const char *name);
1721 (we don't allow values that can't be contained in a C integer.)
1723 This form is typical of the primitive <bf/ODR/ functions. They are named
1724 after the type of data that they encode or decode. They take an
1726 stream, an indirect reference to the type in question, and an
1727 <tt/optional/ flag (corresponding to the OPTIONAL keyword of ASN.1) as
1728 parameters. They all return an integer value of either one or zero.
1729 When you use the primitive functions to construct encoders for complex
1730 types of your own, you should follow this model as well. This
1731 ensures that your new types can be reused as elements in yet more
1734 The <tt/o/ parameter should obviously refer to a properly
1736 stream of the right type (encoding/decoding/printing) for the
1737 operation that you wish to perform.
1739 When encoding or printing, the function first looks at *<tt/p/. If
1741 pointer pointed to by <tt/p/) is a null pointer, this is taken to mean that
1742 the data element is absent. If the <tt/optional/ parameter is nonzero,
1743 the function will return one (signifying success) without any further
1744 processing. If the <tt/optional/ is zero, an internal error flag is
1745 set in the ODR stream, and the function will return 0. No further
1746 operations can be carried out on the stream without a call to
1747 the function <tt/odr_reset()/.
1749 If *<tt/p/ is not a null pointer, it is expected to point to an instance of
1750 the data type. The data will be subjected to the encoding rules, and
1751 the result will be placed in the buffer held by the ODR stream.
1753 The other ASN.1 primitives have similar functions that operate in
1760 int odr_bool(ODR o, bool_t **p, int optional, const char *name);
1772 int odr_null(ODR o, bool_t **p, int optional, const char *name);
1775 In this case, the value of **p is not important. If *p is different
1776 from the null pointer, the null value is present, otherwise it's
1783 typedef struct odr_oct
1790 int odr_octetstring(ODR o, Odr_oct **p, int optional, const char *name);
1793 The <tt/buf/ field should point to the character array that holds the
1794 octetstring. The <tt/len/ field holds the actual length, while the
1796 field gives the size of the allocated array (not of interest to you,
1797 in most cases). The character array need not be null terminated.
1799 To make things a little easier, an alternative is given for string
1800 types that are not expected to contain embedded NULL characters (eg.
1804 int odr_cstring(ODR o, char **p, int optional, const char *name);
1807 Which encoded or decodes between OCTETSTRING representations and
1808 null-terminates C strings.
1810 Functions are provided for the derived string types, eg:
1813 int odr_visiblestring(ODR o, char **p, int optional, const char *name);
1820 int odr_bitstring(ODR o, Odr_bitmask **p, int optional, const char *name);
1823 The opaque type <tt/Odr_bitmask/ is only suitable for holding relatively brief bit
1824 strings, eg. for options fields, etc. The constant
1825 <tt/ODR_BITMASK_SIZE/
1826 multiplied by 8 gives the maximum possible number of bits.
1828 A set of macros are provided for manipulating the
1833 void ODR_MASK_ZERO(Odr_bitmask *b);
1835 void ODR_MASK_SET(Odr_bitmask *b, int bitno);
1837 void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno);
1839 int ODR_MASK_GET(Odr_bitmask *b, int bitno);
1842 The functions are modelled after the manipulation functions that
1843 accompany the <tt/fd_set/ type used by the <tt/select/(2) call.
1844 <tt/ODR_MASK_ZERO/ should always be called first on a new bitmask, to
1845 initialize the bits to zero.
1847 <sect3>OBJECT IDENTIFIER
1851 int odr_oid(ODR o, Odr_oid **p, int optional, const char *name);
1854 The C OID represenation is simply an array of integers, terminated by
1855 the value -1 (the <tt/Odr_oid/ type is synonymous with the <tt/int/
1856 type). We suggest that you use the OID database module (see section
1857 <ref id="oid" name="Object Identifiers">) to handle object identifiers
1858 in your application.
1860 <sect2>Tagging Primitive Types<label id="tag-prim">
1863 The simplest way of tagging a type is to use the <tt/odr_implicit_tag()/ or
1864 <tt/odr_explicit_tag()/ macros:
1867 int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag, int
1868 optional, const char *name);
1870 int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag,
1871 int optional, const char *name);
1874 To create a type derived from the integer type by implicit tagging, you
1878 MyInt ::= [210] IMPLICIT INTEGER
1881 In the <bf/ODR/ system, this would be written like:
1884 int myInt(ODR o, int **p, int optional, const char *name)
1886 return odr_implicit_tag(o, odr_integer, p,
1887 ODR_CONTEXT, 210, optional, name);
1891 The function <tt/myInt()/ can then be used like any of the primitive
1892 functions provided by ODR. Note that the behavior of
1894 and <tt/odr_implicit()/ macros act exactly the same as the functions they
1895 are applied to - they respond to error conditions, etc, in the same
1896 manner - they simply have three extra parameters. The class parameter may
1897 take one of the values: <tt/ODR_CONTEXT/, <tt/ODR_PRIVATE/,
1898 <tt/ODR_UNIVERSAL/, or
1899 <tt/ODR_APPLICATION/.
1901 <sect2>Constructed Types
1904 Constructed types are created by combining primitive types. The
1906 system only implements the SEQUENCE and SEQUENCE OF constructions
1907 (although adding the rest of the container types should be simple
1908 enough, if the need arises).
1910 For implementing SEQUENCEs, the functions
1913 int odr_sequence_begin(ODR o, void *p, int size, const char *name);
1914 int odr_sequence_end(ODR o);
1919 The <tt/odr_sequence_begin()/ function should be called in the beginning of a
1920 function that implements a SEQUENCE type. Its parameters are the
1922 stream, a pointer (to a pointer to the type you're implementing), and
1923 the <tt/size/ of the type (typically a C structure). On encoding, it
1924 returns 1 if *<tt/p/ is a null pointer. The <tt/size/ parameter is ignored. On
1925 decoding, it returns 1 if the type is found in the data stream.
1927 bytes of memory are allocated, and *<tt/p/ is set to point to this space.
1928 <tt/odr_sequence_end()/ is called at the end of the complex function. Assume
1929 that a type is defined like this:
1932 MySequence ::= SEQUENCE {
1934 boolval BOOLEAN OPTIONAL }
1937 The corresponding ODR encoder/decoder function and the associated data
1938 structures could be written like this:
1941 typedef struct MySequence
1947 int mySequence(ODR o, MySequence **p, int optional, const char *name)
1949 if (odr_sequence_begin(o, p, sizeof(**p), name) == 0)
1950 return optional &ero;&ero; odr_ok(o);
1952 odr_integer(o, &ero;(*p)->intval, 0, "intval") &ero;&ero;
1953 odr_bool(o, &ero;(*p)->boolval, 1, "boolval") &ero;&ero;
1954 odr_sequence_end(o);
1958 Note the 1 in the call to <tt/odr_bool()/, to mark that the sequence
1959 member is optional. If either of the member types had been tagged, the
1960 macros <tt/odr_implicit()/ or <tt/odr_explicit()/ could have been used.
1962 function can be used exactly like the standard functions provided with
1963 <bf/ODR/. It will encode, decode or pretty-print a data value of the
1964 <tt/MySequence/ type. We like to name types with an initial capital, as
1965 done in ASN.1 definitions, and to name the corresponding function with
1966 the first character of the name in lower case. You could, of course,
1967 name your structures, types, and functions any way you please - as
1968 long as you're consistent, and your code is easily readable.
1970 just that - a predicate that returns the state of the stream. It is
1971 used to ensure that the behaviour of the new type is compatible with
1972 the interface of the primitive types.
1974 <sect2>Tagging Constructed Types
1978 NOTE: See section <ref id="tag-prim" name="Tagging Primitive types">
1979 for information on how to tag the primitive types, as well as types
1980 that are already defined.
1983 <sect3>Implicit Tagging
1986 Assume the type above had been defined as
1989 MySequence ::= [10] IMPLICIT SEQUENCE {
1991 boolval BOOLEAN OPTIONAL }
1994 You would implement this in <bf/ODR/ by calling the function
1997 int odr_implicit_settag(ODR o, int class, int tag);
2000 which overrides the tag of the type immediately following it. The
2001 macro <tt/odr_implicit()/ works by calling <tt/odr_implicit_settag()/
2003 before calling the function pointer argument. Your type function could
2007 int mySequence(ODR o, MySequence **p, int optional, const char *name)
2009 if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 ||
2010 odr_sequence_begin(o, p, sizeof(**p), name) == 0)
2011 return optional &ero;&ero; odr_ok(o);
2013 odr_integer(o, &ero;(*p)->intval, 0, "intval") &ero;&ero;
2014 odr_bool(o, &ero;(*p)->boolval, 1, "boolval") &ero;&ero;
2015 odr_sequence_end(o);
2019 The definition of the structure <tt/MySequence/ would be the same.
2021 <sect3>Explicit Tagging
2024 Explicit tagging of constructed types is a little more complicated,
2025 since you are in effect adding a level of construction to the data.
2027 Assume the definition:
2030 MySequence ::= [10] IMPLICIT SEQUENCE {
2032 boolval BOOLEAN OPTIONAL }
2035 Since the new type has an extra level of construction, two new functions
2036 are needed to encapsulate the base type:
2039 int odr_constructed_begin(ODR o, void *p, int class, int tag,
2042 int odr_constructed_end(ODR o);
2045 Assume that the IMPLICIT in the type definition above were replaced
2046 with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which
2047 would be equivalent). The structure definition would look the same,
2048 but the function would look like this:
2051 int mySequence(ODR o, MySequence **p, int optional, const char *name)
2053 if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0)
2054 return optional &ero;&ero; odr_ok(o);
2055 if (o->direction == ODR_DECODE)
2056 *p = odr_malloc(o, sizeof(**p));
2057 if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0)
2059 *p = 0; /* this is almost certainly a protocol error */
2063 odr_integer(o, &ero;(*p)->intval, 0, "intval") &ero;&ero;
2064 odr_bool(o, &ero;(*p)->boolval, 1, "boolval") &ero;&ero;
2065 odr_sequence_end(o) &ero;&ero;
2066 odr_constructed_end(o);
2070 Notice that the interface here gets kind of nasty. The reason is
2071 simple: Explicitly tagged, constructed types are fairly rare in
2072 the protocols that we care about, so the
2073 aesthetic annoyance (not to mention the dangers of a cluttered
2074 interface) is less than the time that would be required to develop a
2075 better interface. Nevertheless, it is far from satisfying, and it's a
2076 point that will be worked on in the future. One option for you would
2077 be to simply apply the <tt/odr_explicit()/ macro to the first function,
2079 have to worry about <tt/odr_constructed_*/ yourself. Incidentally, as you
2080 might have guessed, the <tt/odr_sequence_/ functions are themselves
2081 implemented using the <tt/odr_constructed_/ functions.
2086 To handle sequences (arrays) of a apecific type, the function
2089 int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional),
2090 void *p, int *num, const char *name);
2093 The <tt/fun/ parameter is a pointer to the decoder/encoder
2094 function of the type. <tt/p/ is a pointer to an array of pointers to your
2095 type. <tt/num/ is the number of elements in the array.
2100 MyArray ::= SEQUENCE OF INTEGER
2103 The C representation might be
2106 typedef struct MyArray
2113 And the function might look like
2116 int myArray(ODR o, MyArray **p, int optional, const char *name)
2118 if (o->direction == ODR_DECODE)
2119 *p = odr_malloc(o, sizeof(**p));
2120 if (odr_sequence_of(o, odr_integer, &ero;(*p)->elements,
2121 &ero;(*p)->num_elements, name))
2124 return optional &ero;&ero; odr_ok(o);
2131 The choice type is used fairly often in some ASN.1 definitions, so
2132 some work has gone into streamlining its interface.
2134 CHOICE types are handled by the function:
2137 int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp,
2141 The <tt/arm/ array is used to describe each of the possible types that the
2142 CHOICE type may assume. Internally in your application, the CHOICE
2143 type is represented as a discriminated union. That is, a C union
2144 accompanied by an integer (or enum) identifying the active 'arm' of
2145 the union. <tt/whichp/ is a pointer to the union discriminator. When
2146 encoding, it is examined to determine the current type. When decoding,
2147 it is set to reference the type that was found in the input stream.
2149 The Odr_arm type is defined thus:
2152 typedef struct odr_arm
2163 The interpretation of the fields are:
2166 <tag/tagmode/Either <tt/ODR_IMPLICIT/, <tt/ODR_EXPLICIT/, or <tt/ODR_NONE/
2169 <tag/class, tag/The class and tag of the type (-1 if no tagging is
2171 <tag/which/The value of the discriminator that corresponds to
2172 this CHOICE element. Typically, it will be a #defined constant, or
2174 <tag/fun/A pointer to a function that implements the type of
2175 the CHOICE member. It may be either a standard <bf/ODR/ type or a type
2176 defined by yourself.
2177 <tag/name/Name of tag.
2180 A handy way to prepare the array for use by the <tt/odr_choice()/ function
2182 define it as a static, initialized array in the beginning of your
2183 decoding/encoding function. Assume the type definition:
2186 MyChoice ::= CHOICE {
2188 tagged [99] IMPLICIT INTEGER,
2193 Your C type might look like
2196 typedef struct MyChoice
2213 And your function could look like this:
2216 int myChoice(ODR o, MyChoice **p, int optional, const char *name)
2218 static Odr_arm arm[] =
2220 {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"},
2221 {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer,
2223 {-1, -1, -1, MyChoice_other, odr_boolean, "other"},
2227 if (o->direction == ODR_DECODE)
2228 *p = odr_malloc(o, sizeof(**p);
2230 return optional &ero;&ero; odr_ok(o);
2232 if (odr_choice(o, arm, &ero;(*p)->u, &ero;(*p)->which), name)
2235 return optional &ero;&ero; odr_ok(o);
2239 In some cases (say, a non-optional choice which is a member of a sequence),
2240 you can &dquot;embed&dquot; the union and its discriminator in the structure
2241 belonging to the enclosing type, and you won't need to fiddle with
2242 memory allocation to create a separate structure to wrap the
2243 discriminator and union.
2245 The corresponding function is somewhat nicer in the Sun XDR interface.
2246 Most of the complexity of this interface comes from the possibility of
2247 declaring sequence elements (including CHOICEs) optional.
2249 The ASN.1 specifictions naturally requires that each member of a
2250 CHOICE have a distinct tag, so they can be told apart on decoding.
2251 Sometimes it can be useful to define a CHOICE that has multiple types
2252 that share the same tag. You'll need some other mechanism, perhaps
2253 keyed to the context of the CHOICE type. In effect, we would like to
2254 introduce a level of context-sensitiveness to our ASN.1 specification.
2255 When encoding an internal representation, we have no problem, as long
2256 as each CHOICE member has a distinct discriminator value. For
2257 decoding, we need a way to tell the choice function to look for a
2258 specific arm of the table. The function
2261 void odr_choice_bias(ODR o, int what);
2264 provides this functionality. When called, it leaves a notice for the
2265 next call to <tt/odr_choice()/ to be called on the decoding
2266 stream <tt/o/ that only the <tt/arm/ entry with a <tt/which/ field
2267 equal to <tt/what/ should be tried.
2269 The most important application (perhaps the only one, really) is in
2270 the definition of application-specific EXTERNAL encoders/decoders
2271 which will automatically decode an ANY member given the direct or
2277 The protocol modules are suffering somewhat from a lack of diagnostic
2278 tools at the moment. Specifically ways to pretty-print PDUs that
2279 aren't recognized by the system. We'll include something to this end
2280 in a not-too-distant release. In the meantime, what we do when we get
2281 packages we don't understand is to compile the ODR module with
2282 <tt/ODR_DEBUG/ defined. This causes the module to dump tracing
2283 information as it processes data units. With this output and the
2284 protocol specification (Z39.50), it is generally fairly easy to see
2287 <sect>The COMSTACK Module<label id="comstack">
2293 subsystem provides a transparent interface to different types of transport
2294 stacks for the exchange of BER-encoded data. At present, the
2295 RFC1729 method (BER over TCP/IP), and Peter Furniss' XTImOSI
2296 stack are supported, but others may be added in time. The philosophy of the
2297 module is to provide a simple interface by hiding unused options and
2298 facilities of the underlying libraries. This is always done at the risk
2299 of losing generality, and it may prove that the interface will need
2302 The interface is implemented in such a fashion that only the
2303 sub-layers constructed to the transport methods that you wish to
2304 use in your application are linked in.
2306 You will note that even though simplicity was a goal in the design,
2307 the interface is still orders of magnitudes more complex than the
2308 transport systems found in many other packages. One reason is that
2309 the interface needs to support the somewhat different requirements of
2310 the different lower-layer communications stacks; another important reason is
2311 that the interface seeks to provide a more or less industrial-strength
2312 approach to asynchronous event-handling. When no function is allowed
2313 to block, things get more complex - particularly on the server
2314 side. We urge you to have a look at the demonstration client and server
2315 provided with the package. They are meant to be easily readable and
2316 instructive, while still being at least moderately useful.
2318 <sect1>Common Functions
2320 <sect2>Managing Endpoints
2324 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
2327 Creates an instance of the protocol stack - a communications endpoint.
2330 parameter determines the mode of communication. At present, the
2335 are recognized. The function returns a null-pointer if a system error
2336 occurs. The <tt/blocking/ parameter should be one if you wish the
2337 association to operate in blocking mode, zero otherwise. The
2338 <tt/protocol/ field should be one of <tt/PROTO_SR/ or <tt/PROTO_Z3950/.
2341 int cs_close(COMSTACK handle);
2344 Closes the connection (as elegantly as the lower layers will permit),
2345 and releases the resouces pointed to by the
2349 should not be referenced again after this call.
2353 We really need a soft disconnect, don't we?
2356 <sect2>Data Exchange
2360 int cs_put(COMSTACK handle, char *buf, int len);
2365 down the wire. In blocking mode, this function will return only when a
2366 full buffer has been written, or an error has occurred. In nonblocking
2367 mode, it's possible that the function will be unable to send the full
2368 buffer at once, which will be indicated by a return value of 1. The
2369 function will keep track of the number of octets already written; you
2370 should call it repeatedly with the same values of <tt/buf/ and
2371 <tt/len/, until the buffer has been transmitted. When a full buffer
2372 has been sent, the function will return 0 for success. -1 indicates
2373 an error condition (see below).
2376 int cs_get(COMSTACK handle, char **buf, int *size);
2379 Receives a PDU from the peer. Returns the number of bytes
2380 read. In nonblocking mode, it is possible that not all of the packet can be
2381 read at once. In this case, the function returns 1. To simplify the
2382 interface, the function is
2383 responsible for managing the size of the buffer. It will be reallocated
2384 if necessary to contain large packages, and will sometimes be moved
2385 around internally by the subsystem when partial packages are read. Before
2388 for the fist time, the buffer can be initialized to the null pointer,
2389 and the length should also be set to 0 - cs_get will perform a
2391 on the buffer for you. When a full buffer has been read, the size of
2392 the package is returned (which will always be greater than 1). -1
2393 indicates an error condition.
2400 int cs_more(COMSTACK handle);
2403 The <tt/cs_more()/ function should be used in conjunction with
2407 The <tt/cs_get()/ function will sometimes (notably in the TCP/IP mode) read
2408 more than a single protocol package off the network. When this
2409 happens, the extra package is stored by the subsystem. After calling
2410 <tt/cs_get()/, and before waiting for more input,
2411 You should always call
2413 to check if there's a full protocol package already read. If
2417 can be used to immediately fetch the new package. For the
2419 subsystem, the function should always return 0, but if you want your
2420 stuff to be protocol independent, you should use it.
2423 NOTE: The <tt/cs_more()/
2424 function is required because the
2426 does not provide a way of separating individual PDUs, short of
2427 partially decoding the BER. Some other implementations will carefully
2428 nibble at the packet by calling
2430 several times. This was felt to be too inefficient (or at least
2431 clumsy) - hence the call for this extra function.
2435 int cs_look(COMSTACK handle);
2438 This function is useful when you're operating in nonblocking
2441 tells you there's something happening on the line. It returns one of
2442 the following values:
2445 <tag/CS_NONE/No event is pending. The data found on the line was
2446 not a complete package.
2447 <tag/CS_CONNECT/A response to your connect request has been received. Call
2449 to process the event and to finalize the connection establishment.
2450 <tag/CS_DISCON/The other side has closed the connection (or maybe
2452 request - but do we care? Maybe later). Call
2453 <tt/cs_close/ To close your end of the association as well.
2454 <tag/CS_LISTEN/A connect request has been received. Call
2456 to process the event.
2457 <tag/CS_DATA/There's data to be found on the line. Call
2464 You should be aware that even if
2466 tells you that there's an event event pending, the corresponding
2467 function may still return and tell you there was nothing to be found.
2468 This means that only part of a package was available for reading. The
2469 same event will show up again, when more data has arrived.
2473 int cs_fileno(COMSTACK h);
2476 Returns the file descriptor of the association. Use this when
2477 file-level operations on the endpoint are required (<tt/select/(2)
2478 operations, specifically).
2484 int cs_connect(COMSTACK handle, void *address);
2487 Initiate a connection with the target at <tt/address/ (more on
2488 addresses below). The function will return 0 on success, and 1 if
2489 the operation does not complete immediately (this will only
2490 happen on a nonblocking endpoint). In this case, use
2491 <tt/cs_rcvconnect/ to complete the operation, when <tt/select/(2)
2492 reports input pending on the association.
2495 int cs_rcvconnect(COMSTACK handle);
2498 Complete a connect operation initiated by <tt/cs_connect()/. It will
2499 return 0 on success; 1 if the operation has not yet completed (in this
2500 case, call the function again later); -1 if an error has occured.
2505 To establish a server under the <tt/inetd/ server, you can use
2508 COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
2512 The <it/socket/ parameter is an established socket (when your
2513 application is invoked from <tt/inetd/, the socket will typically be
2514 0. The following parameters are identical to the ones for
2518 int cs_bind(COMSTACK handle, void *address, int mode)
2521 Binds a local address to the endpoint. Read about addresses below. The
2522 <tt/mode/ parameter should be either <tt/CS_CLIENT/ or <tt/CS_SERVER/.
2525 int cs_listen(COMSTACK handle, char *addr, int *addrlen);
2528 Call this to process incoming events on an endpoint that has been
2529 bound in listening mode. It will return 0 to indicate that the connect
2530 request has been received, 1 to signal a partial reception, and -1 to
2531 indicate an error condition.
2534 COMSTACK cs_accept(COMSTACK handle);
2537 This finalises the server-side association establishment, after
2538 cs_listen has completed successfully. It returns a new connection
2539 endpoint, which represents the new association. The application will
2540 typically wish to fork off a process to handle the association at this
2541 point, and continue listen for new connections on the old <tt/handle/.
2543 You can use the call
2546 char *cs_addrstr(COMSTACK);
2549 on an established connection to retrieve the hostname of the remote host.
2551 <it>NOTE: You may need to use this function with some care if your
2552 name server service is slow or unreliable</it>
2557 The low-level format of the addresses are different depending on the
2558 mode of communication you have chosen. A function is provided by each
2559 of the lower layers to map a user-friendly string-form address to the
2560 binary form required by the lower layers.
2563 struct sockaddr_in *tcpip_strtoaddr(char *str);
2565 struct netbuf *mosi_strtoaddr(char *str);
2568 The format for TCP/IP addresses is straightforward:
2571 <host> [ ':' <portnum> ]
2574 The <tt/hostname/ can be either a domain name or an IP address. The
2575 port number, if omitted, defaults to 210.
2577 For OSI, the format is
2580 [ <t-selector> '/' ] <host> [ ':' <port> ]
2583 The transport selector is given as an even number of hex digits.
2585 You'll note that the address format for the OSI mode are just a subset
2586 of full presentation addresses. We use presentation addresses because
2587 xtimosi doesn't, in itself, allow access to the X.500 Directory
2588 service. We use a limited form, because we haven't yet come across an
2589 implementation that used more of the elements of a full p-address. It
2590 is a fairly simple matter to add the rest of the elements to the
2591 address format as needed, however: Xtimosi <it/does/ support the full
2592 P-address structure.
2594 In both transport modes, the special hostname &dquot;@&dquot; is mapped
2595 to any local address (the manifest constant INADDR_ANY). It is used
2596 to establish local listening endpoints in the server role.
2598 When a connection has been established, you can use
2601 char cs_addrstr(COMSTACK h);
2604 to retrieve the host name of the peer system. The function returns a pointer
2605 to a static area, which is overwritten on the next call to the function.
2608 NOTE: We have left the issue of X.500 name-to-address mapping open, for the
2609 moment. It would be a simple matter to provide a table-based mapping,
2610 if desired. Alternately, we could use the X.500 client-function that
2611 is provided with the ISODE (although this would defeat some of the
2612 purpose of using ThinOSI in the first place. We have been told that it
2613 should be within the realm of the possible to implement a lightweight
2614 implementation of the necessary X.500 client capabilities on top of
2615 ThinOSI. This would be the ideal solution, we feel. On the other hand, it
2616 still remains to be seen just what role the Directory will play in a world
2617 populated by ThinOSI and other pragmatic solutions.
2623 All functions return -1 if an error occurs. Typically, the functions
2624 will return 0 on success, but the data exchange functions
2625 (<tt/cs_get/, <tt/cs_put/, <tt/cs_more/)
2626 follow special rules. Consult their descriptions.
2628 When a function (including the data exchange functions) reports an
2632 to determine the cause of the
2633 problem. The function
2636 void cs_perror(COMSTACK handle char *message);
2643 argument, along with a system message, to
2645 Use the character array
2648 extern const char *cs_errlist[];
2651 to get hold of the message, if you want to process it differently.
2655 const char *cs_stackerr(COMSTACK handle);
2658 Returns an error message from the lower layer, if one has been
2661 <sect1>Enabling OSI Communication
2663 <sect2>Installing Xtimosi
2665 Although you will have to download Peter Furniss' XTI/mOSI
2666 implementation for yourself, we've tried to make the integration as
2669 The latest version of xtimosi will generally be under
2672 ftp://pluto.ulcc.ac.uk/ulcc/thinosi/xtimosi/
2675 When you have downloaded and unpacked the archive, it will (we assume)
2676 have created a directory called <tt/xtimosi/. We suggest that you place
2677 this directory <it/in the same directory/ where you unpacked the
2679 distribution. This way, you shouldn't have to fiddle with the
2680 makefiles of <tt/YAZ/ beyond uncommenting a few lines.
2682 Go to <tt>xtimosi/src</tt>, and type &dquot;<tt/make libmosi.a/&dquot;.
2683 This should generally
2684 create the library, ready to use.
2689 The currently available release of xtimosi has some inherent
2690 problems that make it disfunction on certain platforms - eg. the
2691 Digital OSF/1 workstations. It is supposedly primarily a
2692 compiler problem, and we hope to see a release that is generally
2693 portable. While we can't guarantee that it can be brought to work
2694 on your platform, we'll be happy to talk to you about problems
2695 that you might see, and relay information to the author of the
2696 software. There are some signs that the <bf/gcc/ compiler is more
2697 likely to produce a fully functional library, but this hasn't
2698 been verified (we think that the problem is limited to the use
2699 of hexadecimal escape-codes used in strings, which are silently
2700 ignored by some compilers).
2704 A problem has been encountered in the communication with
2705 ISODE-based applications. If the ISODE presentation-user calls
2706 <tt/PReadRequest()/ with a timeout value different from <tt/OK/ or <tt/NOTOK/,
2707 he will get an immediate TIMEOUT abort when receiving large (>2041
2708 bytes, which is the SPDU-size that the ISODE likes to work with) packages
2709 from an xtimosi-based implementation (probably most
2710 other implementations as well, in fact). It seems to be a flaw in the
2711 ISODE API, and the workaround (for ISODE users) is to either not
2712 use an explicit timeout (switching to either blocking or
2713 nonblocking mode), or to check that the timer really has expired
2714 before closing the connection.
2717 The next step in the installation is to modify the makefile in the toplevel
2719 directory. The place to change is in the top of the file, and is
2720 clearly marked with a comment.
2722 Now run <tt/make/ in the <bf/YAZ/ toplevel directory (do a
2723 &dquot;<tt/make clean/&dquot; first, if the
2724 system has been previously made without OSI support). Use the <bf/YAZ/
2726 and <bf/client/ demo programs to verify that OSI communication works OK. Then,
2727 you can go ahead and try to talk to other implementations.
2730 NOTE: Our interoperability experience is limited to version
2731 7 of the Nordic SR-Nett package, which has had several
2732 protocol errors fixed from the earlier releases. If you have
2733 problems or successes in interoperating with other
2734 implementations, we'd be glad to hear about it, or to help
2735 you make things work, as our resources allow.
2738 If you write your own applications based on <bf/YAZ/, and you wish to
2739 include OSI support, the procedure is equally simple. You should
2740 include the <tt/xmosi.h/ header file in addition to <tt/comstack.h/.
2742 will define the manifest constant <tt/mosi_type/, which you should pass
2743 to the <tt/cs_create()/ function. In
2744 addition, you should use the function <tt/mosi_strtoaddr()/ rather than
2745 <tt/tcpip_strtoaddr()/ when you need to prepare an address.
2747 When you link your application, you should include (after the
2749 library) the <tt/libmosi.a/ library, and the <tt/librfc.a/ library
2751 <bf/YAZ/ (for OSI transport).
2753 As always, it can be very useful, if not essential, to have a look at the
2754 example applications
2755 to see how things are done.
2757 <sect2>OSI Transport
2760 Xtimosi requires an implementation of the OSI transport service under
2761 the X/OPEN XTI API. We provide an implementation of the RFC1006
2762 encapsulation of OSI/TP0 in TCP/IP (through the Berkeley Sockets API),
2763 as an independent part of <bf/YAZ/ (it's found under the <tt/rfc1006/
2764 directory). If you have access to an OSI transport provider under XTI,
2765 you should be able to make that work too, although it may require
2766 tinkering with the <tt/mosi_strtoaddr()/ function.
2768 <sect2>Presentation Context Management
2771 To simplify the implementation, we use Peter Furniss' alternative (PRF)
2773 for the Control of the presentation negotiation phase. This format
2774 is enabled by default when you
2777 The current version of <bf/YAZ/ does <it/not/ support presentation-layer
2778 negotiation of response record formats. The primary reason is that we
2779 have had access to no other SR or Z39.50 implementations over OSI that
2781 method. Secondarily, we believe that the EXPLAIN facility is a superior
2782 mechanism for relaying target capabilities in this respect. This is not to
2783 say that we have no intentions of supporting presentation context
2784 negotiation - we have just hitherto given it a lower priority than other
2785 aspects of the protocol.
2787 One thing is certain: The addition of this capability to <bf/YAZ/ should
2788 have only a minimal impact on existing applications, and on the
2789 interface to the software in general. Most likely, we will add an extra
2790 layer of interface to the processing of EXPLAIN records, which will
2791 convert back and forth between <tt/oident/ records (see section
2792 <ref id="oid" name="Object Identifiers">) and direct or indirect
2793 references, given the current association setup. Implementations based
2794 on any of the higher-level interfaces will most likely not have to be
2797 <sect1>Summary and Synopsis
2801 #include <comstack.h>
2803 #include <tcpip.h> /* this is for TCP/IP support */
2804 #include <xmosi.h> /* and this is for mOSI support */
2806 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
2808 COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
2811 int cs_bind(COMSTACK handle, int mode);
2813 int cs_connect(COMSTACK handle, void *address);
2815 int cs_rcvconnect(COMSTACK handle);
2817 int cs_listen(COMSTACK handle);
2819 COMSTACK cs_accept(COMSTACK handle);
2821 int cs_put(COMSTACK handle, char *buf, int len);
2823 int cs_get(COMSTACK handle, char **buf, int *size);
2825 int cs_more(COMSTACK handle);
2827 int cs_close(COMSTACK handle);
2829 int cs_look(COMSTACK handle);
2831 struct sockaddr_in *tcpip_strtoaddr(char *str);
2833 struct netbuf *mosi_strtoaddr(char *str);
2835 extern int cs_errno;
2837 void cs_perror(COMSTACK handle char *message);
2839 const char *cs_stackerr(COMSTACK handle);
2841 extern const char *cs_errlist[];
2844 <sect>Making an IR Interface for Your Database with YAZ<label id="server">
2850 NOTE: If you aren't into documentation, a good way to learn how the
2851 backend interface works is to look at the backend.h file. Then,
2852 look at the small dummy-server in server/ztest.c. Finally, you can
2853 have a look at the seshigh.c file, which is where most of the
2854 logic of the frontend server is located. The backend.h file also
2855 makes a good reference, once you've chewed your way through
2856 the prose of this file.
2859 If you have a database system that you would like to make available by
2860 means of Z39.50/SR, <bf/YAZ/ basically offers your two options. You
2861 can use the APIs provided by the <bf/ASN/, <bf/ODR/, and <bf/COMSTACK/
2863 create and decode PDUs, and exchange them with a client. Using this
2864 low-level interface gives you access to all fields and options of the
2865 protocol, and you can construct your server as close to your existing
2866 database as you like. It is also a fairly involved process, requiring
2867 you to set up an event-handling mechanism, protocol state machine,
2868 etc. To simplify server implementation, we have implemented a compact
2869 and simple, but reasonably full-functioned server-frontend that will
2870 handle most of the protocol mechanics, while leaving you to
2871 concentrate on your database interface.
2874 NOTE: The backend interface was designed in anticipation of a specific
2875 integration task, while still attempting to achieve some degree of
2876 generality. We realise fully that there are points where the
2877 interface can be improved significantly. If you have specific
2878 functions or parameters that you think could be useful, send us a
2879 mail (or better, sign on to the mailing list referred to in the
2880 toplevel README file). We will try to fit good suggestions into future
2881 releases, to the extent that it can be done without requiring
2882 too many structural changes in existing applications.
2885 <sect1>The Database Frontend
2888 We refer to this software as a generic database frontend. Your
2889 database system is the <it/backend database/, and the interface between
2890 the two is called the <it/backend API/. The backend API consists of a
2891 small number of function prototypes and structure definitions. You are
2892 required to provide the <bf/main()/ routine for the server (which can be
2893 quite simple), as well as functions to match each of the prototypes.
2894 The interface functions that you write can use any mechanism you like
2895 to communicate with your database system: You might link the whole
2896 thing together with your database application and access it by
2897 function calls; you might use IPC to talk to a database server
2898 somewhere; or you might link with third-party software that handles
2899 the communication for you (like a commercial database client library).
2900 At any rate, the functions will perform the tasks of:
2903 <item>Initialization.
2905 <item>Fetching records.
2906 <item>Scanning the database index (if you wish to implement SCAN).
2909 (more functions will be added in time to support as much of
2910 Z39.50-1995 as possible).
2912 Because the model where pipes or sockets are used to access the backend
2913 database is a fairly common one, we have added a mechanism that allows this
2914 communication to take place asynchronously. In this mode, the frontend
2915 server doesn't have to block while the backend database is processing
2916 a request, but can wait for additional PDUs from the client.
2918 <sect1>The Backend API
2921 The headers files that you need to use the interface are in the
2922 include/ directory. They are called <tt/statserv.h/ and <tt/backend.h/. They
2923 will include other files from the <tt/include/ directory, so you'll
2924 probably want to use the -I option of your compiler to tell it where
2925 to find the files. When you run <tt/make/ in the toplevel <bf/YAZ/ directory,
2926 everything you need to create your server is put the lib/libyaz.a
2927 library. If you want OSI as well, you'll also need to link in the
2928 <tt/libmosi.a/ library from the xtimosi distribution (see the mosi.txt
2929 file), a well as the <tt>lib/librfc.a</tt> library (to provide OSI transport
2932 <sect1>Your main() Routine
2935 As mentioned, your <bf/main()/ routine can be quite brief. If you want to
2936 initialize global parameters, or read global configuration tables,
2937 this is the place to do it. At the end of the routine, you should call
2940 <tscreen><verb>int statserv_main(int argc, char **argv);</verb></tscreen>
2942 <bf/Statserv_main/ will establish listening sockets according to the
2943 parameters given. When connection requests are received, the event
2944 handler will typically <bf/fork()/ to handle the new request. If you do use
2945 global variables, you should be aware, then, that these cannot be
2946 shared between associations, unless you explicitly disallow forking by
2947 command line parameters (we advise against this for any purposes
2948 except debugging, as a crash or hang in the server process will affect
2949 all users currently signed on to the server).
2951 The server provides a mechanism for controlling some of its behavior
2952 without using command-line options. The function
2955 statserv_options_block *statserv_getcontrol(void);
2958 Will return a pointer to a <tt/struct statserv_options_block/ describing
2959 the current default settings of the server. The structure contains
2963 <tag/int dynamic/A boolean value, which determines whether the server
2964 will fork on each incoming request (TRUE), or not (FALSE). Default is
2966 <tag/int loglevel/Set this by ORing the constants defined in include/log.h.
2967 <tag/char logfile[ODR_MAXNAME+1]/File for diagnostic output
2968 (&dquot;&dquot;: stderr).
2969 <tag/char apdufile[ODR_MAXNAME+1]/Name of file for logging incoming and
2970 outgoing APDUs (&dquot;&dquot;: don't log APDUs, &dquot;-&dquot;: <tt/stderr/).
2971 <tag/char default_listen[1024]/Same form as the command-line
2972 specification of listener address. &dquot;&dquot;: no default listener address.
2973 Default is to listen at &dquot;tcp:@:9999&dquot;. You can only
2974 specify one default listener address in this fashion.
2975 <tag/enum oid_proto default_proto;/Either <tt/PROTO_SR/ or <tt/PROTO_Z3950/.
2976 Default is <tt/PROTO_Z39_50/.
2977 <tag/int idle_timeout;/Maximum session idletime, in minutes. Zero indicates
2978 no (infinite) timeout. Default is 120 minutes.
2979 <tag/int maxrecordsize;/Maximum permissible record (message) size. Default
2980 is 1Mb. This amount of memory will only be allocated if a client requests a
2981 very large amount of records in one operation (or a big record). Set it
2983 if you are worried about resource consumption on your host system.
2984 <tag/char configname[ODR_MAXNAME+1]/Passed to the backend when a
2985 new connection is received.
2986 <tag/char setuid[ODR_MAXNAME+1]/Set user id to the user specified,
2987 after binding the listener addresses.
2990 The pointer returned by <tt/statserv_getcontrol/ points to a static area.
2991 You are allowed to change the contents of the structure, but the
2992 changes will not take effect before you call
2995 void statserv_setcontrol(statserv_options_block *block);
2998 Note that you should generally update this structure <it/before/ calling
2999 <tt/statserv_main()/.
3001 <sect1>The Backend Functions
3004 For each service of the protocol, the backend interface declares one or
3005 two functions. You are required to provide implementations of the
3006 functions representing the services that you wish to implement.
3008 <tscreen><verb>bend_initresult *bend_init(bend_initrequest *r);</verb></tscreen>
3010 This function is called once for each new connection request, after
3011 a new process has been forked, and an initRequest has been received
3012 from the client. The parameter and result structures are defined as
3016 typedef struct bend_initrequest
3021 typedef struct bend_initresult
3023 int errcode; /* 0==OK */
3024 char *errstring; /* system error string or NULL */
3025 void *handle; /* private handle to the backend module */
3030 The <tt/configname/ of <tt/bend_initrequest/ is currently always set to
3031 &dquot;default-config&dquot;. We haven't had use for putting anything special in
3032 the initrequest yet, but something might go there if the need arises
3033 (account/password info would be obvious).
3035 In general, the server frontend expects that the <tt/bend_*result/
3036 pointer that you return is valid at least until the next call to a
3037 <tt/bend_* function/. This applies to all of the functions described
3038 herein. The parameter structure passed to you in the call belongs to
3039 the server frontend, and you should not make assumptions about its
3040 contents after the current function call has completed. In other
3041 words, if you want to retain any of the contents of a request
3042 structure, you should copy them.
3044 The <tt/errcode/ should be zero if the initialization of the backend went
3045 well. Any other value will be interpreted as an error. The
3046 <tt/errstring/ isn't used in the current version, but one option
3047 would be to stick it
3048 in the initResponse as a VisibleString. The <tt/handle/ is the most
3049 important parameter. It should be set to some value that uniquely
3050 identifies the current session to the backend implementation. It is
3051 used by the frontend server in any future calls to a backend function.
3052 The typical use is to set it to point to a dynamically allocated state
3053 structure that is private to your backend module.
3057 bend_searchresult *bend_search(void *handle, bend_searchrequest *r,
3059 bend_searchresult *bend_searchresponse(void *handle);
3061 typedef struct bend_searchrequest
3063 char *setname; /* name to give to this set */
3064 int replace_set; /* replace set, if it already exists */
3065 int num_bases; /* number of databases in list */
3066 char **basenames; /* databases to search */
3067 Z_Query *query; /* query structure */
3068 } bend_searchrequest;
3070 typedef struct bend_searchresult
3072 int hits; /* number of hits */
3073 int errcode; /* 0==OK */
3074 char *errstring; /* system error string or NULL */
3075 } bend_searchresult;
3079 The first thing to notice about the search request interface (as well
3080 as all of the following requests), is that it consists of two separate
3081 functions. The idea is to provide a simple facility for
3082 asynchronous communication with the backend server. When a
3083 searchrequest comes in, the server frontend will fill out the
3084 <tt/bend_searchrequest/ tructure, and call the <tt/bend_search
3085 function/. The <tt/fd/
3086 argument will point to an integer variable. If you are able to do
3087 asynchronous I/O with your database server, you should set *<tt/fd/ to the
3088 file descriptor you use for the communication, and return a null
3089 pointer. The server frontend will then <tt/select()/ on the *<tt/fd/,
3091 <tt/bend_searchresult/ when it sees that data is available. If you don't
3092 support asynchronous I/O, you should return a pointer to the
3093 <tt/bend_searchresult/ immediately, and leave *<tt/fd/ untouched. This
3094 construction is common to all of the <tt/bend_/ functions (except
3095 <tt/bend_init/). Note that you can choose to support this facility in none,
3096 any, or all of the <tt/bend_/ functions, and you can respond
3097 differently on each request at run-time. The server frontend will
3100 The <tt/bend_searchrequest/ is a fairly close approximation of a protocol
3101 searchRequest PDU. The <tt/setname/ is the resultSetName from the protocol. You
3102 are required to establish a mapping between the set name and whatever
3103 your backend database likes to use. Similarly, the <tt/replace_set/ is a
3104 boolean value corresponding to the resultSetIndicator field in the
3105 protocol. <tt>Num_bases/basenames</tt> is a length of/array of character
3106 pointers to the database names provided by the client. The <tt/query/ is the
3107 full query structure as defined in the protocol ASN.1 specification.
3108 It can be either of the possible query types, and it's up to you to
3109 determine if you can handle the provided query type. Rather than
3110 reproduce the C interface here, we'll refer you to the structure
3111 definitions in the file <tt>include/proto.h</tt>. If you want to look at the
3112 attributeSetId OID of the RPN query, you can either match it against
3113 your own internal tables, or you can use the <tt/oid_getentbyoid/ function
3114 provided by <bf/YAZ/.
3116 The result structure contains a number of hits, and an
3117 <tt>errcode/errstring</tt> pair. If an error occurs during the search, or if
3118 you're unhappy with the request, you should set the errcode to a value
3119 from the BIB-1 diagnostic set. The value will then be returned to the
3120 user in a nonsurrogate diagnostic record in the response. The
3121 <tt/errstring/, if provided, will go in the addinfo field. Look at the
3122 protocol definition for the defined error codes, and the suggested
3123 uses of the addinfo field.
3127 bend_fetchresult *bend_fetch(void *handle, bend_fetchrequest *r,
3129 bend_fetchresult *bend_fetchresponse(void *handle);
3131 typedef struct bend_fetchrequest
3133 char *setname; /* set name */
3134 int number; /* record number */
3136 } bend_fetchrequest;
3138 typedef struct bend_fetchresult
3140 char *basename; /* name of database that provided record */
3141 int len; /* length of record */
3142 char *record; /* record */
3143 int last_in_set; /* is it? */
3145 int errcode; /* 0==success */
3146 char *errstring; /* system error string or NULL */
3152 NOTE: The <tt/bend_fetchresponse()/ function is not yet supported
3153 in this version of the software. Your implementation of <tt/bend_fetch()/
3154 should always return a pointer to a <tt/bend_fetchresult/.
3157 The frontend server calls <tt/bend_fetch/ when it needs database records to
3158 fulfill a searchRequest or a presentRequest. The <tt/setname/ is simply the
3159 name of the result set that holds the reference to the desired record.
3160 The <tt/number/ is the offset into the set (with 1 being the first record
3161 in the set). The <tt/format/ field is the record format requested by the
3162 client (See section <ref id="oid" name="Object Identifiers">). The value
3163 <tt/VAL_NONE/ indicates that the client did not request a specific format.
3164 The <tt/stream/ argument is an <bf/ODR/ stream which should be used for
3165 allocating space for structured data records. The stream will be reset when
3166 all records have been assembled, and the response package has been transmitted.
3167 For unstructured data, the backend is responsible for maintaining a static
3168 or dynamic buffer for the record between calls.
3170 In the result structure, the <tt/basename/ is the name of the
3171 database that holds the
3172 record. <tt/Len/ is the length of the record returned, in bytes, and
3174 is a pointer to the record. <tt/Last_in_set/ should be nonzero only if the
3175 record returned is the last one in the given result set. <tt/Errcode/ and
3176 <tt/errstring/, if given, will currently be interpreted as a global error
3177 pertaining to the set, and will be returned in a
3178 nonSurrogateDiagnostic.
3180 <it>NOTE: This is silly. Add a flag to say which is which.</it>
3182 If the <tt/len/ field has the value -1, then <tt/record/ is assumed to point
3183 to a constructed data type. The <tt/format/ field will be used to determine
3184 which encoder should be used to serialize the data.
3187 NOTE: If your backend generates structured records, it should use
3188 <tt/odr_malloc()/ on the provided stream for allocating data: This allows
3189 the frontend server to keep track of the record sizes.
3192 The <tt/format/ field is mapped to an object identifier in the direct
3193 reference of the resulting EXTERNAL representation of the record.
3195 <it>NOTE: The current version of <bf/YAZ/ only supports the direct reference
3200 bend_deleteresult *bend_delete(void *handle, bend_deleterequest *r,
3202 bend_deleteresult *bend_deleteresponse(void *handle);
3204 typedef struct bend_deleterequest
3207 } bend_deleterequest;
3209 typedef struct bend_deleteresult
3211 int errcode; /* 0==success */
3212 char *errstring; /* system error string or NULL */
3213 } bend_deleteresult;
3218 NOTE: The &dquot;delete&dquot; function is not yet supported
3219 in this version of the software.
3223 NOTE: The delete set function definition is rather primitive, mostly
3225 have had no practical need for it as of yet. If someone wants
3226 to provide a full delete service, we'd be happy to add the
3227 extra parameters that are required. Are there clients out there
3228 that will actually delete sets they no longer need?
3233 bend_scanresult *bend_scan(void *handle, bend_scanrequest *r,
3235 bend_scanresult *bend_scanresponse(void *handle);
3237 typedef struct bend_scanrequest
3239 int num_bases; /* number of elements in databaselist */
3240 char **basenames; /* databases to search */
3241 Z_AttributesPlusTerm *term;
3242 int term_position; /* desired index of term in result list */
3243 int num_entries; /* number of entries requested */
3246 typedef struct bend_scanresult
3267 NOTE: The <tt/bend_scanresponse()/ function is not yet supported
3268 in this version of the software. Your implementation of <tt/bend_scan()/
3269 should always return a pointer to a <tt/bend_scanresult/.
3272 <sect1>Application Invocation
3275 The finished application has the following
3276 invocation syntax (by way of <tt/statserv_main()/):
3279 appname [-szSu -a apdufile -l logfile -v loglevel]
3280 [listener ...]
3286 <tag/-a/APDU file. Specify a file for dumping PDUs (for diagnostic purposes).
3287 The special name &dquot;-&dquot; sends output to <tt/stderr/.
3289 <tag/-S/Don't fork on connection requests. This is good for debugging, but
3290 not recommended for real operation: Although the server is
3291 asynchronous and non-blocking, it can be nice to keep a software
3292 malfunction (okay then, a crash) from affecting all current users.
3294 <tag/-s/Use the SR protocol.
3296 <tag/-z/Use the Z39.50 protocol (default). These two options complement
3297 eachother. You can use both multiple times on the same command
3298 line, between listener-specifications (see below). This way, you
3299 can set up the server to listen for connections in both protocols
3300 concurrently, on different local ports.
3302 <tag/-l/The logfile.
3304 <tag/-v/The log level. Use a comma-separated list of members of the set
3305 {fatal,debug,warn,log,all,none}.
3306 <tag/-u/Set user ID. Sets the real UID of the server process to that of the
3307 given user. It's useful if you aren't comfortable with having the
3308 server run as root, but you need to start it as such to bind a
3311 <tag/-w/Working directory.
3312 <tag/-i/Use this when running from the <tt/inetd/ server.
3314 <tag/-t/Idle session timeout, in minutes.
3316 <tag/-k/Maximum record size/message size, in kilobytes.
3320 A listener specification consists of a transport mode followed by a
3321 colon (:) followed by a listener address. The transport mode is
3322 either <tt/osi/ or <tt/tcp/.
3324 For TCP, an address has the form
3327 hostname | IP-number [: portnumber]
3330 The port number defaults to 210 (standard Z39.50 port).
3332 For osi, the address form is
3335 [t-selector /] hostname | IP-number [: portnumber]
3338 The transport selector is given as a string of hex digits (with an even
3339 number of digits). The default port number is 102 (RFC1006 port).
3347 osi:0402/dbserver.osiworld.com:3000
3351 In both cases, the special hostname &dquot;@&dquot; is mapped to
3352 the address INADDR_ANY, which causes the server to listen on any local
3353 interface. To start the server listening on the registered ports for
3354 Z39.50 and SR over OSI/RFC1006, and to drop root privileges once the
3355 ports are bound, execute the server like this (from a root shell):
3358 my-server -u daemon tcp:@ -s osi:@
3361 You can replace <tt/daemon/ with another user, eg. your own account, or
3362 a dedicated IR server account. <tt/my-server/ should be the name of your
3363 server application. You can test the procedure with the <tt/ztest/
3366 <sect1>Summary and Synopsis
3370 #include <backend.h>
3372 bend_initresult *bend_init(bend_initrequest *r);
3374 bend_searchresult *bend_search(void *handle, bend_searchrequest *r,
3377 bend_searchresult *bend_searchresponse(void *handle);
3379 bend_fetchresult *bend_fetch(void *handle, bend_fetchrequest *r,
3382 bend_fetchresult *bend_fetchresponse(void *handle);
3384 bend_scanresult *bend_scan(void *handle, bend_scanrequest *r, int *fd);
3386 bend_scanresult *bend_scanresponse(void *handle);
3388 bend_deleteresult *bend_delete(void *handle, bend_deleterequest *r,
3391 bend_deleteresult *bend_deleteresponse(void *handle);
3393 void bend_close(void *handle);
3396 <sect>Future Directions
3399 The software has been
3400 successfully ported to the Mac as well as Windows NT/95 - we'd like to
3401 test those ports better and make sure they work as they should.
3403 We have a new and better version of the frontend server on the drawing
3404 board. Resources and external commitments will govern when we'll be
3405 able to do something real with it. Fetures should include greater
3406 flexibility, greter support for access/resource control, and easy
3407 support for Explain (possibly with Zebra as an extra database engine).
3409 We now support all PDUs of Z39.50-1995. If there is one of the
3410 supporting structures that you need but can't find in the prt*.h
3411 files, send us a note; it may be on its way.
3413 The 'retrieval' module needs to be finalized and documented. We think
3414 it can form a useful resource for people dealing with complex record
3415 structures, but for now, you'll mostly have to chew through the code
3416 yourself to make use of it. Not acceptable.
3418 Other than that, YAZ generally moves in the directions which appear to
3419 make the most people happy (including ourselves, as prime users of the
3420 software). If there's something you'd like to see in here, then drop
3421 us a note and let's see what we can come up with.
3425 <sect1>Index Data Copyright
3428 Copyright © 1995-2000 Index Data.
3430 Permission to use, copy, modify, distribute, and sell this software and
3431 its documentation, in whole or in part, for any purpose, is hereby granted,
3434 1. This copyright and permission notice appear in all copies of the
3435 software and its documentation. Notices of copyright or attribution
3436 which appear at the beginning of any file must remain unchanged.
3438 2. The names of Index Data or the individual authors may not be used to
3439 endorse or promote products derived from this software without specific
3440 prior written permission.
3442 THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND,
3443 EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY
3444 WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
3445 IN NO EVENT SHALL INDEX DATA BE LIABLE FOR ANY SPECIAL, INCIDENTAL,
3446 INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES
3447 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR
3448 NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF
3449 LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
3452 <sect1>Additional Copyright Statements
3455 The optional CCL query language interpreter is covered by the following
3458 Copyright © 1995, the EUROPAGATE consortium (see below).
3460 The EUROPAGATE consortium members are:
3462 University College Dublin
3463 Danmarks Teknologiske Videnscenter
3464 An Chomhairle Leabharlanna
3465 Consejo Superior de Investigaciones Cientificas
3467 Permission to use, copy, modify, distribute, and sell this software and
3468 its documentation, in whole or in part, for any purpose, is hereby granted,
3471 1. This copyright and permission notice appear in all copies of the
3472 software and its documentation. Notices of copyright or attribution
3473 which appear at the beginning of any file must remain unchanged.
3475 2. The names of EUROPAGATE or the project partners may not be used to
3476 endorse or promote products derived from this software without specific
3477 prior written permission.
3479 3. Users of this software (implementors and gateway operators) agree to
3480 inform the EUROPAGATE consortium of their use of the software. This
3481 information will be used to evaluate the EUROPAGATE project and the
3482 software, and to plan further developments. The consortium may use
3483 the information in later publications.
3485 4. Users of this software agree to make their best efforts, when
3486 documenting their use of the software, to acknowledge the EUROPAGATE
3487 consortium, and the role played by the software in their work.
3489 THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND,
3490 EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY
3491 WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
3492 IN NO EVENT SHALL THE EUROPAGATE CONSORTIUM OR ITS MEMBERS BE LIABLE
3493 FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF
3494 ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA
3495 OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND
3496 ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE
3497 USE OR PERFORMANCE OF THIS SOFTWARE.
3499 <sect>About Index Data
3502 Index Data is a consulting and software-development enterprise that
3503 specialises in library and information management systems. Our
3504 interests and expertise span a broad range of related fields, and one
3505 of our primary, long-term objectives is the development of a powerful
3506 information management
3507 system with open network interfaces and hypermedia capabilities.
3509 We make this software available free of charge, on a fairly unrestrictive
3510 license; as a service to the networking community, and to further the
3511 development of quality software for open network communication.
3513 We'll be happy to answer questions about the software, and about ourselves
3519 DK-2200 Copenhagen N
3524 Phone: +45 3536 3672
3526 Email: info@indexdata.dk
3529 The <it>Hacker's Jargon File</it> has the following to say about the
3531 prefix &dquot;YA&dquot; in the name of a software product.
3534 Yet Another. adj. 1. Of your own work: A
3535 humorous allusion often used in titles to acknowledge that the
3536 topic is not original, though the content is. As in &dquot;Yet Another
3537 AI Group&dquot; or &dquot;Yet Another Simulated Annealing Algorithm&dquot;.
3539 others' work: Describes something of which there are already far
projects / yaz-moved-to-github.git / blob