1 <!doctype linuxdoc system>
3 <title>YAZ User's Guide and Reference
4 <author><htmlurl url="http://www.indexdata.dk/" name="Index Data">, <tt><htmlurl url="mailto:info@indexdata.dk" name="info@indexdata.dk"></>
7 This document is the programmer's guide and reference to the YAZ
8 package. YAZ is a compact toolkit that provides access to the
9 Z39.50/SR protocol, as well as a set of higher-level tools for
10 implementing the server and client roles, respectively.
11 The documentation can be used on its own, or as a reference when looking
12 at the example applications provided with the package.
20 The <bf/YAZ/ toolkit offers several different levels of access to the
21 Z39.50 and SR protocols. The level that you need to use depends on
22 your requirements, and the role (server or client) that you
25 The basic level, which is independent of the role, consists of three
29 <item><bf/ASN/, which provides a C representation of the Z39.50/SR protocol
31 <item><bf/ODR/, which encodes and decodes the packages according to the BER
33 <item><bf/COMSTACK/, which exchanges the encoded packages with a peer process
37 The ASN module represents the ASN.1 definition of the
38 SR/Z39.50 protocol. It establishes a set of type and
39 structure definitions, with one structure for each of the top-level
40 PDUs, and one structure or type for each of the contained ASN.1 types.
41 For primitive types, or other types that are defined by the ASN.1
42 standard itself (such as the EXTERNAL type), the C representation is provided
43 by the <bf/ODR/ (Open Data Representation) subsystem.
45 <bf/ODR/ is a basic mechanism for representing an
46 ASN.1 type in the C programming language, and for implementing BER
47 encoders and decoders for values of that type. The types defined in
49 module generally have the prefix <tt/Z_/, and a suffix corresponding to
50 the name of the type in the ASN.1
51 specification of the protocol (generally Z39.50-1995). In the case of
52 base types (those originating in the ASN.1 standard itself), the prefix
53 <tt/Odr_/ is sometimes seen. Either way, look for
54 the actual definition in either <tt/proto.h/ (for the types from the
55 protocol), <tt/odr.h/ (for the primitive ASN.1 types, or <tt/odr_use.h/ (for the
56 ASN.1 <it/useful/ types). The <bf/ASN/ library also provides functions (which
57 are, in turn, defined using <bf/ODR/ primitives) for encoding and decoding data
58 values. Their general form is
61 int z_xxx(ODR o, Z_xxx **p, int optional, const char *name);
64 (note the lower-case &dquot;z&dquot; in the function name)
67 NOTE: If you are using the premade definitions of the <bf/ASN/ module, and
68 you are not adding new protocol of your own, the only parts of ODR
69 that you need to worry about are documented in section
70 <ref id="odr-use" name="Using ODR">.
73 When you have created a BER-encoded buffer, you can use the <bf/COMSTACK/
74 subsystem to transmit (or receive) data over the network. The <bf/COMSTACK/
75 module provides simple functions for establishing a connection
76 (passively or actively, depending on the role of your application),
77 and for exchanging BER-encoded PDUs over that connection. When you
78 create a connection endpoint, you need to specify what transport to
79 use (OSI or TCP/IP), and which protocol you want to use (SR or
80 Z39.50). For the remainer of the connection's lifetime, you don't have
81 to worry about the underlying transport protocol at all - the <bf/COMSTACK/
82 will ensure that the correct mechanism is used.
84 We call the combined interfaces to <bf/ODR/, <bf/ASN/, and <bf/COMSTACK/ the service
85 level API. It's the API that most closely models the Z39.50/SR
86 service/protocol definition, and it provides unlimited access to all
87 fields and facilities of the protocol definitions.
89 The reason that the <bf/YAZ/ service-level API is a conglomerate of the
90 APIs from three different submodules is twofold. First, we wanted to allow the
91 user a choice of different options for each major task. For instance,
92 if you don't like the protocol API provided by <bf/ODR//<bf/ASN/, you
93 can use SNACC or BERUtils instead, and still have the benefits of the
94 transparent transport approach of the <bf/COMSTACK/ module. Secondly,
95 we realise that you may have to fit the toolkit into an existing
96 event-processing structure, in a way that
97 is incompatible with the <bf/COMSTACK/ interface or some other part of <bf/YAZ/.
99 <sect>Compilation and Installation
102 The latest version of the software will generally be found at
105 http://ftp.indexdata.dk/pub/yaz/
108 We have tried our best to keep the software portable, and on many
109 platforms, you should be able to compile everything with little or no changes.
110 So far, the software has been ported
111 to the following platforms with little or no difficulties.
121 <item>Data General DG/UX (with some CFLAGS tinkering)
125 <item>Non-unix systems
127 <item>Apple Macintosh (using the Codewarrior programming environment and the
128 GUSI socket libraries)
129 <item>MS Windows 95/NT (Win32)
134 If you move the software to other platforms, we'd be grateful if you'd
135 let us know about it. If you run into difficulties, we will try to help if we
136 can, and if you solve the problems, we would be happy to
137 include your fixes in the next release. So far, we have mostly avoided
138 #ifdefs for individual platforms, and we'd like to keep it that
139 way as far as it makes sense.
141 We maintain a mailing-list for the purpose of announcing new releases and
142 bug-fixes, as well as general discussion. Subscribe by sending mail to
143 <tt/yaz-request@indexdata.dk/. General questions and problems can be
144 directed at <tt/yaz-help@indexdata.dk/, or the address given at the top
150 Note that if your system doesn't have a native ANSI C compiler, you may
151 have to acquire one separately. We recommend gcc.
153 For UNIX we use GNU configure to create Makefiles for YAZ.
154 Generally it should be sufficient to run configure without options:
160 The configure script attempts to use use the C compiler specified by
161 the <tt/CC/ environment variable. If not set, GNU C will be used
162 if it is available. The <tt/CFLAGS/ environment variable holds options
163 to be passed to the C compiler. If you're using Bourne-compatible shell
164 you may pass something like this to use a particular C compiler with
165 optimization enabled:
167 CC=/opt/ccs/bin/cc CFLAGS=-O ./configure
170 To customize <bf/YAZ/ the configure script also accepts a set of options.
171 The most important are:
174 <tag><tt>-</tt><tt>-prefix </tt>path</tag> Specifies installation prefix. This is
175 only needed if you run <tt>make install</tt> later to perform a
176 "system" installation. The prefix is <tt>/usr/local</tt> if not
179 <tag><tt>-</tt><tt>-enable-comp </tt></tag> YAZ will be built using
180 the ASN.1 compiler for YAZ (default). If you wish to use the
181 old decoders (in sub directory asn) use <tt>--disable-comp</tt> instead.
183 <tag><tt>-</tt><tt>-enable-threads</tt></tag> YAZ will be built using
184 POSIX threads. Specifically, <tt>_REENTRANT</tt> will be defined
188 When configured, build the software by typing:
193 The following files are generated by the make process:
195 <tag><tt>lib/libyaz.a</tt></tag> The <bf/YAZ/ programmers' library.
197 <tag><tt>ztest/yaz-ztest</tt></tag> A test Z39.50 server.
199 <tag><tt>client/yaz-client</tt></tag> A command mode Z39.50 client.
201 <tag><tt>yaz-config</tt></tag> A Bourne-shell script that holds build
202 settings for <bf/YAZ/.
204 <tag><tt>yaz-comp</tt></tag> The ASN.1 compiler for YAZ. Requires the
205 Tcl Shell, tclsh, in current path to work.
209 If you wish to install <bf/YAZ/ in system directories such as /usr/local/bin,
210 /usr/local/lib) you can type:
216 You probably need to have root access in order to perform this.
217 You must specify the <tt>--prefix</tt> option for configure if you
218 going to install in anything but /usr/local/.
220 If you wish to perform an un-installation of YAZ you use:
226 This will only work if you haven't reconfigured YAZ (and therefore
227 changed installation prefix). Note that uninstall will not
228 remove directories created by make install, e.g.
229 <tt>/usr/local/include/yaz</tt>.
234 <bf/YAZ/ is shipped with "makefiles" for the NMAKE tool that comes
237 Start an MS-DOS prompt and switch the sub directory <tt>WIN</tt> where
238 the file <tt>makefile</tt> is located. Customize the installation
239 by editing the <tt>makefile</tt> file (for example by using notepad).
241 The following summarises the most important settings in that file:
244 <tag><tt>NEW_Z3950</tt></tag> If 1, the auto-generated decoder/encoders
245 for Z39.50 as written by the ASN.1 compiler will be used. If 0, the old
246 decoders for Z39.50 will be used. Note, when 1, the setting TCL should
247 point to the Tcl shell on your system.
248 <tag><tt>DEBUG</tt></tag> If set to 1, the software is
249 compiled with debugging libraries. If set to 0, the software
250 is compiled with release (non-debugging) libraries.
253 When satisfied with the settings in the makefile type
258 The following is generated upon successful compilation:
260 <tag><tt>bin/yaz.dll</tt></tag> A multithreaded DLL with everything
261 except the frontend server library.
262 <tag><tt>lib/yaz.lib</tt></tag> An import library for <tt>yaz.dll</tt>.
263 <tag><tt>lib/server.lib</tt></tag> The frontend server library.
264 <tag><tt>bin/yaz-ztest.exe</tt></tag> A test Z39.50 server.
265 <tag><tt>bin/yaz-client.exe</tt></tag> A command mode Z39.50 client.
273 The <bf/ASN/ module provides you with a set of C struct definitions for the
274 various PDUs of the protocol, as well as for the complex types
275 appearing within the PDUs. For the primitive data types, the C
276 representation often takes the form of an ordinary C language type,
277 such as <tt/int/. For ASN.1 constructs that have no direct
278 representation in C, such as general octet strings and bit strings,
279 the <bf/ODR/ module (see section <ref id="odr" name="ODR">) provides auxiliary
282 <sect1>Preparing PDUs
285 A structure representing a complex ASN.1 type doesn't in itself contain the
286 members of that type. Instead, the structure contains <it/pointers/ to
287 the members of the type. This is necessary, in part, to allow a mechanism for
288 specifying which of the optional structure (SEQUENCE) members are
289 present, and which are not. It follows that you will need to somehow
290 provide space for the individual members of the structure, and
291 set the pointers to refer to the members.
293 The conversion routines don't care how you allocate and maintain your
294 C structures - they just follow the pointers that you provide.
295 Depending on the complexity of your application, and your personal
296 taste, there are at least three different approaches that you may take
297 when you allocate the structures.
301 You can use static or automatic local variables in the function that
302 prepares the PDU. This is a simple approach, and it provides the most
303 efficient form of memory management. While it works well for flat
304 PDUs like the InitReqest, it will generally not be sufficient for say,
305 the generation of an arbitrarily complex RPN query structure.
307 You can individually create the structure and its members using the
308 <tt/malloc/(2) function. If you want to ensure that the data is freed when
309 it is no longer needed, you will have to define a function that
310 individually releases each member of a structure before freeing the
313 You can use the <tt/odr_malloc()/ function (see section <ref
315 name="Using ODR"> for details). When you use <tt/odr_malloc()/, you can release
317 allocated data in a single operation, independent of any pointers and
318 relations between the data. <tt/odr_malloc()/ is based on a
319 &dquot;nibble-memory&dquot;
320 scheme, in which large portions of memory are allocated, and then
321 gradually handed out with each call to <tt/odr_malloc()/. The next time you
322 call <tt/odr_reset()/, all of the memory allocated since the last call is
323 recycled for future use (actually, it is placed on a free-list).
325 You can combine all of the methods described here. This will often be
326 the most practical approach. For instance, you might use <tt/odr_malloc()/ to
327 allocate an entire structure and some of its elements, while you leave
328 other elements pointing to global or per-session default variables.
331 The <bf/ASN/ module provides an important aid in creating new PDUs. For
332 each of the PDU types (say, <tt/Z_InitRequest/), a function is provided
333 that allocates and initializes an instance of that PDU type for you.
334 In the case of the InitRequest, the function is simply named
335 <tt/zget_InitRequest()/, and it sets up reasonable default value for all of
336 the mandatory members. The optional members are generally initialized to null
337 pointers. This last aspect is very important: it ensures that if the
338 PDU definitions are extended after you finish your implementation
340 new versions of the protocol, say), you won't get into trouble with
341 uninitialized pointers in your structures. The functions use
343 allocate the PDUs and its members, so you can free everything again with a
344 single call to <tt/odr_reset()/. We strongly recommend that you use the
346 functions whenever you are preparing a PDU (in a C++ API, the
348 functions would probably be promoted to constructors for the
351 The prototype for the individual PDU types generally look like this:
354 Z_<type> *zget_<type>(ODR o);
360 Z_InitRequest *zget_InitRequest(ODR o);
363 The <bf/ODR/ handle should generally be your encoding stream, but it
366 As well as the individual PDU functions, a function <tt/zget_APDU()/ is
367 provided, which allocates a toplevel Z-APDU of the type requested:
370 Z_APDU *zget_APDU(ODR o, int which);
373 The <tt/which/ parameter is (of course) the discriminator belonging to the
374 <tt/Z_APDU/ CHOICE type. All of the interface described here is provided by
375 the <bf/ASN/ module, and you access it through the <tt/proto.h/ header file.
377 <sect1>Object Identifiers<label id="oid">
380 When you refer to object identifiers in your application, you need to
381 be aware that SR and Z39.50 use two different set of OIDs to refer to
382 the same objects. To handle this easily, <bf/YAZ/ provides a utility module
383 to <bf/ASN/ which provides an internal representation of the OIDs used in
384 both protocols. Each oid is described by a structure:
387 typedef struct oident
389 enum oid_proto proto;
390 enum oid_class class;
391 enum oid_value value;
392 int oidsuffix[OID_SIZE];
397 The <tt/proto/ field can be set to either <tt/PROTO_SR/ or
398 <tt/PROTO_Z3950/. The <tt/class/ might be, say, <tt/CLASS_RECSYN/, and the
399 <tt/value/ might be <tt/VAL_USMARC/ for the USMARC record format. Functions
402 int *oid_ent_to_oid(struct oident *ent, int *dst);
403 struct oident *oid_getentbyoid(int *o);
406 are provided to map between object identifiers and database entries.
407 If you store a member of the <tt/oid_proto/ type in your association
408 state information, it's a simple matter, at runtime, to generate the
409 correct OID when you need it. For decoding, you can simply ignore the
410 proto field, or if you're strict, you can verify that your peer is
411 using the OID family from the correct protocol. The <tt/desc/ field is
412 a short, human-readable name for the PDU, useful mainly for diagnostic
416 NOTE: The old function oid_getoidbyent still exists but is
417 not thread safe. Use oid_ent_to_oid instead and pass an array of
422 NOTE: Plans are underway to merge the two protocols into a single
423 definition, with one set of object identifiers. When this happens, the
424 oid module will no longer be required to support protocol
425 independence, but it should still be useful as a simple OID database.
431 In order to achieve extensibility and adaptability to different
432 application domains, the new version of the protocol defines many
433 structures outside of the main ASN.1 specification, referencing them
434 through ASN.1 EXTERNAL constructs. To simplify the construction and access
435 to the externally referenced data, the <bf/ASN/ module defines a
436 specialized version of the EXTERNAL construct, called <tt/Z_External/.
440 typedef struct Z_External
442 Odr_oid *direct_reference;
443 int *indirect_reference;
448 Z_External_single = 0,
450 Z_External_arbitrary,
454 Z_External_explainRecord,
455 Z_External_resourceReport1,
456 Z_External_resourceReport2
464 Odr_any *single_ASN1_type;
465 Odr_oct *octet_aligned;
466 Odr_bitmask *arbitrary;
470 Z_ExplainRecord *explainRecord;
471 Z_ResourceReport1 *resourceReport1;
472 Z_ResourceReport2 *resourceReport2;
480 When decoding, the <bf/ASN/ module will attempt to determine which
481 syntax describes the data by looking at the reference fields
482 (currently only the direct-reference). For ASN.1 structured data, you
483 need only consult the <tt/which/ field to determine the type of data.
484 You can the access the data directly through the union. When
485 constructing data for encoding, you set the union pointer to point to
486 the data, and set the <tt/which/ field accordingly. Remember also to
487 set the direct (or indirect) reference to the correct OID for the data
488 type. For non-ASN.1 data such as MARC records, use the
489 <tt/octet_aligned/ arm of the union.
491 Some servers return ASN.1 structured data values (eg. database
492 records) as BER-encoded records placed in the <tt/octet-aligned/
493 branch of the EXTERNAL CHOICE. The ASN-module will <it/not/
494 automatically decode these records. To help you decode the records in
495 the application, the function
498 Z_ext_typeent *z_ext_gettypebyref(oid_value ref);
501 Can be used to retrieve information about the known, external data
502 types. The function return a pointer to a static area, or NULL, if no
503 match for the given direct reference is found. The <tt/Z_ext_typeent/
507 typedef struct Z_ext_typeent
509 oid_value dref; /* the direct-reference OID value. */
510 int what; /* discriminator value for the external CHOICE */
511 Odr_fun fun; /* decoder function */
515 The <tt/what/ member contains the Z_External union discriminator value
516 for the given type: For the SUTRS record syntax, the value would be
517 <tt/Z_External_sutrs/. The <tt/fun/ member contains a pointer to the
518 function which encodes/decodes the given type. Again, for the SUTRS
519 record syntax, the value of <tt/fun/ would be <tt/z_SUTRS/ (a function
522 If you receive an EXTERNAL which contains an octet-string value that
523 you suspect of being an ASN.1-structured data value, you can use
524 <tt/z_ext_gettypebyref/ to look for the provided direct-reference. If
525 the return value is different from NULL, you can use the provided
526 function to decode the BER string (see section <ref id="odr-use"
529 If you want to <it/send/ EXTERNALs containing ASN.1-structured values
530 in the occtet-aligned branch of the CHOICE, this is possible too.
531 However, on the encoding phase, it requires a somewhat involved
532 juggling around of the various buffers involved.
534 If you need to add new, externally defined data types, you must update
535 the struct above, in the source file <tt/prt-ext.h/, as well as the
536 encoder/decoder in the file <tt/prt-ext.c/. When changing the latter,
537 remember to update both the <tt/arm/ arrary and the list <tt/type_table/,
538 which drives the CHOICE biasing that is necessary to tell the
539 different, structured types apart on decoding.
542 NOTE: Eventually, the EXTERNAL processing will most likely
543 automatically insert the correct OIDs or indirect-refs. First,
544 however, we need to determine how application-context management
545 (specifically the presentation-context-list) should fit into the
549 <sect1>PDU Contents Table
552 We include, for reference, a listing of the fields of each top-level
553 PDU, as well as their default settings.
558 Field Type Default value
560 referenceId Z_ReferenceId NULL
561 protocolVersion Odr_bitmask Empty bitmask
562 options Odr_bitmask Empty bitmask
563 preferredMessageSize int 30*1024
564 maximumRecordSize int 30*1024
565 idAuthentication Z_IdAuthentication NULL
566 implementationId char* "YAZ (id=81)"
567 implementationName char* "Index Data/YAZ"
568 implementationVersion char* YAZ_VERSION
569 userInformationField Z_UserInformation NULL
570 otherInfo Z_OtherInformation NULL
574 Field Type Default value
576 referenceId Z_ReferenceId NULL
577 protocolVersion Odr_bitmask Empty bitmask
578 options Odr_bitmask Empty bitmask
579 preferredMessageSize int 30*1024
580 maximumRecordSize int 30*1024
582 implementationId char* "YAZ (id=81)"
583 implementationName char* "Index Data/YAZ"
584 implementationVersion char* YAZ_VERSION
585 userInformationField Z_UserInformat.. NULL
586 otherInfo Z_OtherInformation NULL
590 Field Type Default value
592 referenceId Z_ReferenceId NULL
593 smallSetUpperBound int 0
594 largeSetLowerBound int 1
595 mediumSetPresentNumber int 0
596 replaceIndicator bool_t TRUE
597 resultSetName char* "default"
598 num_databaseNames int 0
599 databaseNames char** NULL
600 smallSetElementSetNames Z_ElementSetNames NULL
601 mediumSetElementSetNames Z_ElementSetNames NULL
602 preferredRecordSyntax Odr_oid NULL
604 additionalSearchInfo Z_OtherInformation NULL
605 otherInfo Z_OtherInformation NULL
609 Field Type Default value
611 referenceId Z_ReferenceId NULL
613 numberOfRecordsReturned int 0
614 nextResultSetPosition int 0
615 searchStatus bool_t TRUE
616 resultSetStatus int NULL
617 presentStatus int NULL
618 records Z_Records NULL
619 additionalSearchInfo Z_OtherInformation NULL
620 otherInfo Z_OtherInformation NULL
624 Field Type Default value
626 referenceId Z_ReferenceId NULL
627 resultSetId char* "default"
628 resultSetStartPoint int 1
629 numberOfRecordsRequested int 10
631 additionalRanges Z_Range NULL
632 recordComposition Z_RecordComposition NULL
633 preferredRecordSyntax Odr_oid NULL
634 maxSegmentCount int NULL
635 maxRecordSize int NULL
636 maxSegmentSize int NULL
637 otherInfo Z_OtherInformation NULL
641 Field Type Default value
643 referenceId Z_ReferenceId NULL
644 numberOfRecordsReturned int 0
645 nextResultSetPosition int 0
646 presentStatus int Z_PRES_SUCCESS
647 records Z_Records NULL
648 otherInfo Z_OtherInformation NULL
650 Z_DeleteResultSetRequest
651 ------------------------
652 Field Type Default value
654 referenceId Z_ReferenceId NULL
655 deleteFunction int Z_DeleteRequest_list
657 resultSetList char** NULL
658 otherInfo Z_OtherInformation NULL
660 Z_DeleteResultSetResponse
661 -------------------------
662 Field Type Default value
664 referenceId Z_ReferenceId NULL
665 deleteOperationStatus int Z_DeleteStatus_success
667 deleteListStatuses Z_ListStatus** NULL
668 numberNotDeleted int NULL
669 num_bulkStatuses int 0
670 bulkStatuses Z_ListStatus NULL
671 deleteMessage char* NULL
672 otherInfo Z_OtherInformation NULL
676 Field Type Default value
678 referenceId Z_ReferenceId NULL
679 num_databaseNames int 0
680 databaseNames char** NULL
681 attributeSet Odr_oid NULL
682 termListAndStartPoint Z_AttributesPlus... NULL
684 numberOfTermsRequested int 20
685 preferredPositionInResponse int NULL
686 otherInfo Z_OtherInformation NULL
690 Field Type Default value
692 referenceId Z_ReferenceId NULL
694 scanStatus int Z_Scan_success
695 numberOfEntriesReturned int 0
696 positionOfTerm int NULL
697 entries Z_ListEntris NULL
698 attributeSet Odr_oid NULL
699 otherInfo Z_OtherInformation NULL
701 Z_TriggerResourceControlRequest
702 -------------------------------
703 Field Type Default value
705 referenceId Z_ReferenceId NULL
706 requestedAction int Z_TriggerResourceCtrl_resou..
707 prefResourceReportFormat Odr_oid NULL
708 resultSetWanted bool_t NULL
709 otherInfo Z_OtherInformation NULL
711 Z_ResourceControlRequest
712 ------------------------
713 Field Type Default value
715 referenceId Z_ReferenceId NULL
716 suspendedFlag bool_t NULL
717 resourceReport Z_External NULL
718 partialResultsAvailable int NULL
719 responseRequired bool_t FALSE
720 triggeredRequestFlag bool_t NULL
721 otherInfo Z_OtherInformation NULL
723 Z_ResourceControlResponse
724 -------------------------
725 Field Type Default value
727 referenceId Z_ReferenceId NULL
728 continueFlag bool_t TRUE
729 resultSetWanted bool_t NULL
730 otherInfo Z_OtherInformation NULL
732 Z_AccessControlRequest
733 ----------------------
734 Field Type Default value
736 referenceId Z_ReferenceId NULL
737 which enum Z_AccessRequest_simpleForm;
739 otherInfo Z_OtherInformation NULL
741 Z_AccessControlResponse
742 -----------------------
743 Field Type Default value
745 referenceId Z_ReferenceId NULL
746 which enum Z_AccessResponse_simpleForm
748 diagnostic Z_DiagRec NULL
749 otherInfo Z_OtherInformation NULL
753 Field Type Default value
755 referenceId Z_ReferenceId NULL
756 numberOfRecordsReturned int value=0
757 num_segmentRecords int 0
758 segmentRecords Z_NamePlusRecord NULL
759 otherInfo Z_OtherInformation NULL
763 Field Type Default value
765 referenceId Z_ReferenceId NULL
766 closeReason int Z_Close_finished
767 diagnosticInformation char* NULL
768 resourceReportFormat Odr_oid NULL
769 resourceFormat Z_External NULL
770 otherInfo Z_OtherInformation NULL
773 <sect>Supporting Tools
776 In support of the service API - primarily the ASN module, which
777 provides the programmatic interface to the Z39.50 APDUs, YAZ contains
778 a collection of tools that support the development of applications.
780 <sect1>Query Syntax Parsers
783 Since the type-1 (RPN) query structure has no direct, useful string
784 representation, every origin application needs to provide some form of
785 mapping from a local query notation or representation to a
786 <tt/Z_RPNQuery/ structure. Some programmers will prefer to construct
787 the query manually, perhaps using <tt/odr_malloc()/ to simplify memory
788 management. The YAZ distribution includes two separate,
789 query-generating tools that may be of use to you.
791 <sect2>Prefix Query Format
794 Since RPN or reverse polish notation is really just a fancy way of
795 describing a suffix notation format (operator follows operands), it
796 would seem that the confusion is total when we now introduce a prefix
797 notation for RPN. The reason is one of simple laziness - it's somewhat
798 simpler to interpret a prefix format, and this utility was designed
799 for maximum simplicity, to provide a baseline representation for use
800 in simple test applications and scripting environments (like Tcl). The
801 demonstration client included with YAZ uses the PQF.
803 The PQF is defined by the pquery module in the YAZ library. The
804 <tt/pquery.h/ file provides the declaration of the functions
807 Z_RPNQuery *p_query_rpn (ODR o, oid_proto proto, const char *qbuf);
809 Z_AttributesPlusTerm *p_query_scan (ODR o, oid_proto proto,
810 Odr_oid **attributeSetP, const char *qbuf);
812 int p_query_attset (const char *arg);
815 The function <tt/p_query_rpn()/ takes as arguments an <bf/ODR/ stream
816 (see section <ref id="odr" name="The ODR Module">) to provide a memory
817 source (the structure created is released on the next call to
818 <tt/odr_reset()/ on the stream/), a protocol identifier (one of the
819 constants <tt/PROTO_Z3950/ and <tt/PROTO_SR/), an attribute set
820 reference, and finally a null-terminated string holding the query
823 If the parse went well, <tt/p_query_rpn()/ returns a pointer to a
824 <tt/Z_RPNQuery/ structure which can be placed directly into a
825 <tt/Z_SearchRequest/.
827 The <tt/p_query_attset/ specifies which attribute set to use if
828 the query doesn't specify one by the <tt/@attrset/ operator. The
829 <tt/p_query_attset/ returns 0 if the argument is a valid attribute
830 set specifier; otherwise the function returns -1.
832 The grammar of the PQF is as follows:
835 Query ::= [ AttSet ] QueryStruct.
839 QueryStruct ::= { Attribute } Simple | Complex.
841 Attribute ::= '@attr' AttributeType '=' AttributeValue.
843 AttributeType ::= integer.
845 AttributeValue ::= integer.
847 Complex ::= Operator QueryStruct QueryStruct.
849 Operator ::= '@and' | '@or' | '@not' | '@prox' Proximity.
851 Simple ::= ResultSet | Term.
853 ResultSet ::= '@set' string.
855 Term ::= string | '"' string '"'.
857 Proximity ::= Exclusion Distance Ordered Relation WhichCode UnitCode.
859 Exclusion ::= '1' | '0' | 'void'.
861 Distance ::= integer.
863 Ordered ::= '1' | '0'.
865 Relation ::= integer.
867 WhichCode ::= 'known' | 'private' | integer.
869 UnitCode ::= integer.
872 You will note that the syntax above is a fairly faithful
873 representation of RPN, except for the <tt/Attibute/, which has been
874 moved a step away from the term, allowing you to associate one or more
875 attributes with an entire query structure. The parser will
876 automatically apply the given attributes to each term as required.
878 The following are all examples of valid queries in the PQF.
885 @or "dylan" "zimmerman"
889 @or @and bob dylan @set Result-1
891 @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
893 @attr 4=1 @attr 1=4 "self portrait"
895 @prox 0 3 1 2 k 2 dylan zimmerman
898 <sect2>Common Command Language
901 Not all users enjoy typing in prefix query structures and numerical
902 attribute values, even in a minimalistic test client. In the library
903 world, the more intuitive Common Command Language (or ISO 8777) has
904 enjoyed some popularity - especially before the widespread
905 availability of graphical interfaces. It is still useful in
906 applications where you for some reason or other need to provide a
907 symbolic language for expressing boolean query structures.
909 The EUROPAGATE research project working under the Libraries programme
910 of the European Commission's DG XIII has, amongst other useful tools,
911 implemented a general-purpose CCL parser which produces an output
912 structure that can be trivially converted to the internal RPN
913 representation of YAZ (The <tt/Z_RPNQuery/ structure). Since the CCL
914 utility - along with the rest of the software produced by EUROPAGATE -
915 is made freely available on a liberal license, it is included as a
920 The CCL parser obeys the following grammar for the FIND argument.
921 The syntax is annotated by in the lines prefixed by <tt/--/.
924 CCL-Find ::= CCL-Find Op Elements
927 Op ::= "and" | "or" | "not"
928 -- The above means that Elements are separated by boolean operators.
930 Elements ::= '(' CCL-Find ')'
933 | Qualifiers Relation Terms
934 | Qualifiers Relation '(' CCL-Find ')'
935 | Qualifiers '=' string '-' string
936 -- Elements is either a recursive definition, a result set reference, a
937 -- list of terms, qualifiers followed by terms, qualifiers followed
938 -- by a recursive definition or qualifiers in a range (lower - upper).
940 Set ::= 'set' = string
941 -- Reference to a result set
943 Terms ::= Terms Prox Term
945 -- Proximity of terms.
949 -- This basically means that a term may include a blank
951 Qualifiers ::= Qualifiers ',' string
953 -- Qualifiers is a list of strings separated by comma
955 Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
956 -- Relational operators. This really doesn't follow the ISO8777
960 -- Proximity operator
964 The following queries are all valid:
975 (dylan and bob) or set=1
978 Assuming that the qualifiers <tt/ti/, <tt/au/ and <tt/date/
979 are defined we may use:
983 au=(bob dylan and slow train coming)
985 date>1980 and (ti=((self portrait)))
988 <sect3>CCL Qualifiers
991 Qualifiers are used to direct the search to a particular searchable
992 index, such as title (ti) and author indexes (au). The CCL standard
993 itself doesn't specify a particular set of qualifiers, but it does
994 suggest a few short-hand notations. You can customize the CCL parser
995 to support a particular set of qualifiers to relect the current target
996 profile. Traditionally, a qualifier would map to a particular
997 use-attribute within the BIB-1 attribute set. However, you could also
998 define qualifiers that would set, for example, the
1001 Consider a scenario where the target support ranked searches in the
1002 title-index. In this case, the user could specify
1004 ti,ranked=knuth computer
1006 and the <tt/ranked/ would map to structure=free-form-text
1007 (4=105) and the <tt/ti/ would map to title (1=4).
1009 A "profile" with a set predefined CCL qualifiers can be read from a
1010 file. The YAZ client reads its CCL qualifiers from a file named
1011 <tt/default.bib/. Each line in the file has the form:
1013 <it/qualifier-name/ <it/type/=<it/val/ <it/type/=<it/val/ ...
1015 where <it/qualifier-name/ is the name of the qualifier to be used
1016 (eg. <tt/ti/), <it/type/ is a BIB-1 category type and <it/val/ is the
1017 corresponding BIB-1 attribute value. The <it/type/ can be either
1018 numeric or it may be either <tt/u/ (use), <tt/r/ (relation), <tt/p/
1019 (position), <tt/s/ (structure), <tt/t/ (truncation) or <tt/c/
1020 (completeness). The <it/qualifier-name/ <tt/term/ has a special
1021 meaning. The types and values for this definition is used when <it/no/
1022 qualifier is present.
1024 Consider the following definition:
1030 Two qualifiers are defined, <tt/ti/ and <tt/au/. They both set the
1031 structure-attribute to phrase (1). <tt/ti/ sets the use-attribute to
1032 4. <tt/au/ sets the use-attribute to 1. When no qualifiers are used
1033 in the query the structure-attribute is set to free-form-text (105).
1038 All public definitions can be found in the header file <tt/ccl.h/.
1039 A profile identifier is of type <tt/CCL_bibset/. A profile must be
1040 created with the call to the function <tt/ccl_qual_mk/ which returns
1041 a profile handle of type <tt/CCL_bibset/.
1043 To read a file containing qualifier definitions the function
1044 <tt/ccl_qual_file/ may be convenient. This function takes an already
1045 opened <tt/FILE/ handle pointer as argument along with a
1046 <tt/CCL_bibset/ handle.
1048 To parse a simple string with a FIND query use the function
1050 struct ccl_rpn_node *ccl_find_str (CCL_bibset bibset, const char *str,
1051 int *error, int *pos);
1053 which takes the CCL profile (<tt/bibset/) and query (<tt/str/) as
1054 input. Upon successful completion the RPN tree is returned. If an
1055 error eccur, such as a syntax error, the integer pointed to by
1056 <tt/error/ holds the error code and <tt/pos/ holds the offset inside
1057 query string in which the parsing failed.
1059 An english representation of the error may be obtained by calling
1060 the <tt/ccl_err_msg/ function. The error codes are listed in
1063 To convert the CCL RPN tree (type <tt/struct ccl_rpn_node */) to the
1064 Z_RPNQuery of YAZ the function <tt/ccl_rpn_query/ must be used. This
1065 function which is part of YAZ is implemented in <tt/yaz-ccl.c/.
1066 After calling this function the CCL RPN tree is probably no longer
1067 needed. The <tt/ccl_rpn_delete/ destroys the CCL RPN tree.
1069 A CCL profile may be destroyed by calling the <tt/ccl_qual_rm/
1072 The token names for the CCL operators may be changed by setting the
1073 globals (all type <tt/char */) <tt/ccl_token_and/, <tt/ccl_token_or/,
1074 <tt/ccl_token_not/ and <tt/ccl_token_set/.
1075 An operator may have aliases, i.e. there may be more than one name for
1076 the operator. To do this, separate each alias with a space character.
1078 <sect1>Object Identifiers
1081 The basic YAZ representation of an OID is an array of integers,
1082 terminated with the value -1. The <bf/ODR/ module provides two
1083 utility-functions to create and copy this type of data elements:
1086 Odr_oid *odr_getoidbystr(ODR o, char *str);
1089 Creates an OID based on a string-based representation using dots (.)
1090 to separate elements in the OID.
1093 Odr_oid *odr_oiddup(ODR odr, Odr_oid *o);
1096 Creates a copy of the OID referenced by the <it/o/ parameter. Both
1097 functions take an <bf/ODR/ stream as parameter. This stream is used to
1098 allocate memory for the data elements, which is released on a
1099 subsequent call to <tt/odr_reset()/ on that stream.
1101 The <bf/OID/ module provides a higher-level representation of the
1102 family of object identifers which describe the Z39.50 protocol and its
1103 related objects. The definition of the module interface is given in
1104 the <tt/oid.h/ file.
1106 The interface is mainly based on the <tt/oident/ structure. The
1107 definition of this structure looks like this:
1110 typedef struct oident
1115 int oidsuffix[OID_SIZE];
1120 The <it/proto/ field takes one of the values
1127 If you don't care about talking to SR-based implementations (few
1128 exist, and they may become fewer still if and when the ISO SR and ANSI
1129 Z39.50 documents are merged into a single standard), you can ignore
1130 this field on incoming packages, and always set it to PROTO_Z3950
1131 for outgoing packages.
1133 The <it/oclass/ field takes one of the values
1153 corresponding to the OID classes defined by the Z39.50 standard.
1155 Finally, the <it/value/ field takes one of the values
1211 again, corresponding to the specific OIDs defined by the standard.
1213 The <it/desc/ field contains a brief, mnemonic name for the OID in
1219 struct oident *oid_getentbyoid(int *o);
1222 takes as argument an OID, and returns a pointer to a static area
1223 containing an <tt/oident/ structure. You typically use this function
1224 when you receive a PDU containing an OID, and you wish to branch out
1225 depending on the specific OID value.
1230 int *oid_ent_to_oid(struct oident *ent, int *dst);
1233 Takes as argument an <tt/oident/ structure - in which the <it/proto/,
1234 <it/oclass/, and <it/value/ fields are assumed to be set correctly -
1235 and returns a pointer to a the buffer as given by <it/dst/
1237 representation of the corresponding OID. The function returns
1238 NULL and the array dst is unchanged if a mapping couldn't place.
1239 The array <it/dst/ should be at least of size <tt>OID_SIZE</tt>.
1241 The <tt/oid_ent_to_oid()/ function can be used whenever you need to
1242 prepare a PDU containing one or more OIDs. The separation of the
1243 <it/protocol/ element from the remainer of the OID-description makes
1244 it simple to write applications that can communicate with either
1245 Z39.50 or OSI SR-based applications.
1250 oid_value oid_getvalbyname(const char *name);
1253 takes as argument a mnemonic OID name, and returns the <it/value/
1254 field of the first entry in the database that contains the given name
1255 in its <it/desc/ field.
1257 Finally, the module provides the following utility functions, whose
1258 meaning should be obvious:
1261 void oid_oidcpy(int *t, int *s);
1262 void oid_oidcat(int *t, int *s);
1263 int oid_oidcmp(int *o1, int *o2);
1264 int oid_oidlen(int *o);
1268 NOTE: The <bf/OID/ module has been criticized - and perhaps rightly so
1269 - for needlessly abstracting the
1270 representation of OIDs. Other toolkits use a simple
1271 string-representation of OIDs with good results. In practice, we have
1272 found the interface comfortable and quick to work with, and it is a
1273 simple matter (for what it's worth) to create applications compatible
1274 with both ISO SR and Z39.50. Finally, the use of the <tt/oident/
1275 database is by no means mandatory. You can easily create your
1276 own system for representing OIDs, as long as it is compatible with the
1277 low-level integer-array representation of the ODR module.
1280 <sect1>Nibble Memory
1283 Sometimes when you need to allocate and construct a large,
1284 interconnected complex of structures, it can be a bit of a pain to
1285 release the associated memory again. For the structures describing the
1286 Z39.50 PDUs and related structures, it is convenient to use the
1287 memory-management system of the <bf/ODR/ subsystem (see
1288 <ref id="odr-use" name="Using ODR">). However, in some circumstances
1289 where you might otherwise benefit from using a simple nibble memory
1290 management system, it may be impractical to use <tt/odr_malloc()/ and
1291 <bf/odr_reset()/. For this purpose, the memory manager which also
1292 supports the <bf/ODR/ streams is made available in the <bf/NMEM/
1293 module. The external interface to this module is given in the
1296 The following prototypes are given:
1299 NMEM nmem_create(void);
1300 void nmem_destroy(NMEM n);
1301 void *nmem_malloc(NMEM n, int size);
1302 void nmem_reset(NMEM n);
1303 int nmem_total(NMEM n);
1304 void nmem_init(void);
1307 The <tt/nmem_create()/ function returns a pointer to a memory control
1308 handle, which can be released again by <tt/nmem_destroy()/ when no
1309 longer needed. The function <tt/nmem_malloc()/ allocates a block of
1310 memory of the requested size. A call to <tt/nmem_reset()/ or
1311 <tt/nmem_destroy()/ will release all memory allocated on the handle
1312 since it was created (or since the last call to
1313 <tt/nmem_reset()/. The function <tt/nmem_total()/ returns the number
1314 of bytes currently allocated on the handle.
1316 Note that the nibble memory pool is shared amonst threads. Posix
1317 mutex'es and WIN32 Critical sections are introduced to keep the
1318 module thread safe. On WIN32 function <tt/nmem_init()/ initialises
1319 the Critical Section handle and should be called once before any
1320 other nmem function is used.
1322 <sect>The ODR Module<label id="odr">
1327 <bf/ODR/ is the BER-encoding/decoding subsystem of <bf/YAZ/. Care as been taken
1328 to isolate <bf/ODR/ from the rest of the package - specifically from the
1329 transport interface. <bf/ODR/ may be used in any context where basic
1330 ASN.1/BER representations are used.
1332 If you are only interested in writing a Z39.50 implementation based on
1333 the PDUs that are already provided with <bf/YAZ/, you only need to concern
1334 yourself with the section on managing ODR streams (section
1335 <ref id="odr-use" name="Using ODR">). Only if you need to
1336 implement ASN.1 beyond that which has been provided, should you
1337 worry about the second half of the documentation
1338 (section <ref id="odr-prog" name="Programming with ODR">). If you use
1339 one of the higher-level interfaces, you can skip this section entirely.
1341 This is important, so we'll repeat it for emphasis: <it>You do not
1342 need to read section <ref id="odr-prog" name="Programming with ODR"> to
1343 implement Z39.50 with <bf/YAZ/.</it>
1345 If you need a part of the protocol that isn't already in <bf/YAZ/, you
1347 contact the authors before going to work on it yourself: We
1348 might already be working on it. Conversely, if you implement a useful
1349 part of the protocol before us, we'd be happy to include it in a
1352 <sect1>Using ODR<label id="odr-use">
1358 Conceptually, the ODR stream is the source of encoded data in the
1359 decoding mode; when encoding, it is the receptacle for the encoded
1360 data. Before you can use an ODR stream it must be allocated. This is
1361 done with the function
1364 ODR odr_createmem(int direction);
1367 The <tt/odr_createmem()/ function takes as argument one of three manifest
1368 constants: <tt/ODR_ENCODE/, <tt/ODR_DECODE/, or <tt/ODR_PRINT/. An
1370 in only one mode - it is not possible to change its mode once it's
1371 selected. Typically, your program will allocate at least two ODR
1372 streams - one for decoding, and one for encoding.
1374 When you're done with the stream, you can use
1377 void odr_destroy(ODR o);
1380 to release the resources allocated for the stream.
1382 <sect2>Memory Management<label id="memory">
1385 Two forms of memory management take place in the ODR system. The first
1386 one, which has to do with allocating little bits of memory (sometimes
1387 quite large bits of memory, actually) when a protocol package is
1388 decoded, and turned into a complex of interlinked structures. This
1389 section deals with this system, and how you can use it for your own
1390 purposes. The next section deals with the memory management which is
1391 required when encoding data - to make sure that a large enough buffer is
1392 available to hold the fully encoded PDU.
1394 The <bf/ODR/ module has its own memory management system, which is
1395 used whenever memory is required. Specifically, it is used to allocate
1396 space for data when decoding incoming PDUs. You can use the memory
1397 system for your own purposes, by using the function
1400 void *odr_malloc(ODR o, int size);
1403 You can't use the normal <tt/free/(2) routine to free memory allocated
1404 by this function, and <bf/ODR/ doesn't provide a parallel function. Instead,
1408 void odr_reset(ODR o, int size);
1411 when you are done with the
1412 memory: Everything allocated since the last call to <tt/odr_reset()/ is
1413 released. The <tt/odr_reset()/ call is also required to clear up an
1414 error condition on a stream.
1419 int odr_total(ODR o);
1422 returns the number of bytes allocated on the stream since the last call to
1425 The memory subsystem of <bf/ODR/ is fairly efficient at allocating and
1426 releasing little bits of memory. Rather than managing the individual,
1427 small bits of space, the system maintains a freelist of larger chunks
1428 of memory, which are handed out in small bits. This scheme is
1429 generally known as a <it/nibble memory/ system. It is very useful for
1430 maintaing short-lived constructions such as protocol PDUs.
1432 If you want to retain a bit of memory beyond the next call to
1433 <tt/odr_reset()/, you can use the function
1436 ODR_MEM odr_extract_mem(ODR o);
1439 This function will give you control of the memory recently allocated
1440 on the ODR stream. The memory will live (past calls to <tt/odr_reset()/),
1441 until you call the function
1444 void odr_release_mem(ODR_MEM p);
1447 The opaque <tt/ODR_MEM/ handle has no other purpose than referencing
1448 the memory block for you until you want to release it.
1450 You can use <tt/odr_extract_mem()/ repeatedly between allocating data,
1451 to retain individual control of separate chunks of data.
1453 <sect2>Encoding and Decoding Data
1456 When encoding data, the ODR stream will write the encoded octet string
1457 in an internal buffer. To retrieve the data, use the function
1460 char *odr_getbuf(ODR o, int *len, int *size);
1463 The integer pointed to by len is set to the length of the encoded
1464 data, and a pointer to that data is returned. *<tt/size/ is set to the size
1465 of the buffer (unless <tt/size/ is null, signalling that you are
1466 not interested in the size). The next call to a primitive function
1467 using the same ODR stream will overwrite the data, unless a different
1468 buffer has been supplied using the call
1471 void odr_setbuf(ODR o, char *buf, int len, int can_grow);
1474 which sets the encoding (or decoding) buffer used by <tt/o/ to
1476 using the length <tt/len/. Before a call to an encoding function,
1477 you can use <tt/odr_setbuf()/ to provide the stream with an encoding buffer
1478 of sufficient size (length). The <tt/can_grow/ parameter tells the encoding
1479 ODR stream whether it is allowed to use <tt/realloc/(2) to increase the size
1480 of the buffer when necessary. The default condition of a new encoding
1481 stream is equivalent to the results of calling
1484 odr_setbuf(stream, 0, 0, 1);
1487 In this case, the stream will allocate and reallocate memory as
1488 necessary. The stream reallocates memory by repeatedly doubling the
1489 size of the buffer - the result is that the buffer will typically
1490 reach its maximum, working size with only a small number of reallocation
1491 operations. The memory is freed by the stream when the latter is destroyed,
1492 unless it was assigned by the user with the <tt/can_grow/
1493 parameter set to zero (in this case, you are expected to retain
1494 control of the memory yourself).
1496 To assume full control of an encoded buffer, you must first call
1497 <tt/odr_getbuf()/ to fetch the buffer and its length. Next, you should
1498 call <tt/odr_setbuf()/ to provide a different buffer (or a null
1499 pointer) to the stream. In the simplest case, you will reuse the same
1500 buffer over and over again, and you will just need to call
1501 <tt/odr_getbuf()/ after each encoding operation to get the length and
1502 address of the buffer. Note that the stream may reallocate the buffer
1503 during an encoding operation, so it is necessary to retrieve the
1504 correct address after each encoding operation.
1506 It is important to realise that the ODR stream will not release this
1507 memory when you call <tt/odr_reset()/: It will merely update its
1508 internal pointers to prepare for the encoding of a new data value.
1509 When the stream is released by the <tt/odr_destroy()/ function, the
1510 memory given to it by odr_setbuf will be released <it/only/ if the
1511 <tt/can_grow/ parameter to <tt/odr_setbuf()/ was nonzero. The
1512 <tt/can_grow/ parameter, in other words, is a way of signalling who
1513 is to own the buffer, you or the ODR stream. If you never call
1514 <tt/odr_setbuf()/ on your encoding stream, which is typically the
1515 case, the buffer allocated by the stream will belong to the stream by
1518 When you wish to decode data, you should first call <tt/odr_setbuf()/, to
1519 tell the decoding stream where to find the encoded data, and how long
1520 the buffer is (the <tt/can_grow/ parameter is ignored by a decoding
1521 stream). After this, you can call the function corresponding to the
1522 data you wish to decode (eg, <tt/odr_integer()/ odr <tt/z_APDU()/).
1524 Examples of encoding/decoding functions:
1527 int odr_integer(ODR o, int **p, int optional, const char *name);
1529 int z_APDU(ODR o, Z_APDU **p, int optional, const char *name);
1532 If the data is absent (or
1533 doesn't match the tag corresponding to the type), the return value
1534 will be either 0 or 1 depending on the <tt/optional/ flag. If
1536 is 0 and the data is absent, an error flag will be raised in the
1537 stream, and you'll need to call <tt/odr_reset()/ before you can use the
1538 stream again. If <tt/optional/ is nonzero, the pointer <it/pointed to/ by
1539 <tt/p/ will be set to the null value, and the function will return 1.
1540 The <tt/name/ argument is used to pretty-print the tag in question.
1541 It may be set to <tt/NULL/ if pretty-printing is not desired.
1543 If the data value is found where it's expected, the pointer
1544 <it/pointed to/ by the <tt/p/ argument will be set to point to the
1546 space for the type will be allocated and owned by the ODR stream, and
1547 it will live until you call <tt/odr_reset()/ on the stream. You cannot use
1548 <tt/free/(2) to release the memory. You can decode several data elements
1549 (by repeated calls to <tt/odr_setbuf()/ and your decoding function), and
1550 new memory will be allocated each time. When you do call
1552 everything decoded since the last call to <tt/odr_reset()/ will be
1555 The use of the double indirection can be a little confusing at first
1556 (its purpose will become clear later on, hopefully),
1557 so an example is in order. We'll encode an integer value, and
1558 immediately decode it again using a different stream. A useless, but
1559 informative operation.
1562 void do_nothing_useful(int value)
1565 int *valp, *resvalp;
1569 /* allocate streams */
1570 if (!(encode = odr_createmem(ODR_ENCODE)))
1572 if (!(decode = odr_createmem(ODR_DECODE)))
1576 if (odr_integer(encode, &ero;valp, 0, 0) == 0)
1578 printf("encoding went bad\n");
1581 bufferp = odr_getbuf(encode, &ero;len);
1582 printf("length of encoded data is %d\n", len);
1584 /* now let's decode the thing again */
1585 odr_setbuf(decode, bufferp, len);
1586 if (odr_integer(decode, &ero;resvalp, 0, 0) == 0)
1588 printf("decoding went bad\n");
1591 printf("the value is %d\n", *resvalp);
1594 odr_destroy(encode);
1595 odr_destroy(decode);
1599 This looks like a lot of work, offhand. In practice, the ODR streams
1600 will typically be allocated once, in the beginning of your program (or at the
1601 beginning of a new network session), and the encoding and decoding
1602 will only take place in a few, isolated places in your program, so the
1603 overhead is quite manageable.
1608 The encoding/decoding functions all return 0 when an error occurs.
1609 Until you call <tt/odr_reset()/, you cannot use the stream again, and
1610 any function called will immediately return 0.
1612 To provide information to the programmer or administrator, the
1616 void odr_perror(ODR o, char *message);
1619 is provided, which prints the <tt/message/ argument to <tt/stderr/
1620 along with an error message from the stream.
1622 You can also use the function
1625 int odr_geterror(ODR o);
1628 to get the current error number from the screen. The number will be
1629 one of these constants:
1632 <tag/OMEMORY/Memory allocation failed.
1633 <tag/OSYSERR/A system- or library call has failed. The standard
1634 diagnostic variable <tt/errno/
1635 should be examined to determine the actual error.
1636 <tag/OSPACE/No more space for encoding. This will only occur when the user has
1637 explicitly provided a buffer for an encoding stream without allowing
1638 the system to allocate more space.
1639 <tag/OREQUIRED/This is a common protocol error; A required data element was
1641 encoding or decoding.
1642 <tag/OUNEXPECTED/An unexpected data element was found during decoding.
1643 <tag/OOTHER/Other error. This is typically an indication of misuse of
1644 the <bf/ODR/ system by the programmer, and also that the diagnostic
1645 system isn't as good as it should be, yet.
1648 The character string array
1651 char *odr_errlist[]
1654 can be indexed by the error code to obtain a human-readable
1655 representation of the problem.
1657 <sect2>Summary and Synopsis
1663 ODR odr_createmem(int direction);
1665 void odr_destroy(ODR o);
1667 void odr_reset(ODR o);
1669 char *odr_getbuf(ODR o, int *len);
1671 void odr_setbuf(ODR o, char *buf, int len);
1673 void *odr_malloc(ODR o, int size);
1675 ODR_MEM odr_extract_mem(ODR o);
1677 void odr_release_mem(ODR_MEM r);
1679 int odr_geterror(ODR o);
1681 void odr_perror(char *message);
1683 extern char *odr_errlist[];
1686 <sect1>Programming with ODR<label id="odr-prog">
1689 The API of <bf/ODR/ is designed to reflect the structure of ASN.1, rather
1690 than BER itself. Future releases may be able to represent data in
1691 other external forms.
1693 The interface is based loosely on that of the Sun Microsystems XDR routines.
1694 Specifically, each function which corresponds to an ASN.1 primitive
1695 type has a dual function. Depending on the settings of the ODR
1696 stream which is supplied as a parameter, the function may be used
1697 either to encode or decode data. The functions that can be built
1698 using these primitive functions, to represent more complex datatypes, share
1699 this quality. The result is that you only have to enter the definition
1700 for a type once - and you have the functionality of encoding, decoding
1701 (and pretty-printing) all in one unit. The resulting C source code is
1702 quite compact, and is a pretty straightforward representation of the
1703 source ASN.1 specification. Although no ASN.1 compiler is supplied
1704 with <bf/ODR/ at this time, it shouldn't be too difficult to write one, or
1705 perhaps even to adapt an existing compiler to output <bf/ODR/ routines
1706 (not surprisingly, writing encoders/decoders using <bf/ODR/ turns out
1709 In many cases, the model of the XDR functions works quite well in this
1711 others, it is less elegant. Most of the hassle comes from the optional
1712 SEQUENCE memebers which don't exist in XDR.
1714 <sect2>The Primitive ASN.1 Types
1717 ASN.1 defines a number of primitive types (many of which correspond
1719 primitive types in structured programming languages, such as C).
1724 The <bf/ODR/ function for encoding or decoding (or printing) the ASN.1
1725 INTEGER type looks like this:
1728 int odr_integer(ODR o, int **p, int optional, const char *name);
1731 (we don't allow values that can't be contained in a C integer.)
1733 This form is typical of the primitive <bf/ODR/ functions. They are named
1734 after the type of data that they encode or decode. They take an
1736 stream, an indirect reference to the type in question, and an
1737 <tt/optional/ flag (corresponding to the OPTIONAL keyword of ASN.1) as
1738 parameters. They all return an integer value of either one or zero.
1739 When you use the primitive functions to construct encoders for complex
1740 types of your own, you should follow this model as well. This
1741 ensures that your new types can be reused as elements in yet more
1744 The <tt/o/ parameter should obviously refer to a properly
1746 stream of the right type (encoding/decoding/printing) for the
1747 operation that you wish to perform.
1749 When encoding or printing, the function first looks at *<tt/p/. If
1751 pointer pointed to by <tt/p/) is a null pointer, this is taken to mean that
1752 the data element is absent. If the <tt/optional/ parameter is nonzero,
1753 the function will return one (signifying success) without any further
1754 processing. If the <tt/optional/ is zero, an internal error flag is
1755 set in the ODR stream, and the function will return 0. No further
1756 operations can be carried out on the stream without a call to
1757 the function <tt/odr_reset()/.
1759 If *<tt/p/ is not a null pointer, it is expected to point to an instance of
1760 the data type. The data will be subjected to the encoding rules, and
1761 the result will be placed in the buffer held by the ODR stream.
1763 The other ASN.1 primitives have similar functions that operate in
1770 int odr_bool(ODR o, bool_t **p, int optional, const char *name);
1782 int odr_null(ODR o, bool_t **p, int optional, const char *name);
1785 In this case, the value of **p is not important. If *p is different
1786 from the null pointer, the null value is present, otherwise it's
1793 typedef struct odr_oct
1800 int odr_octetstring(ODR o, Odr_oct **p, int optional, const char *name);
1803 The <tt/buf/ field should point to the character array that holds the
1804 octetstring. The <tt/len/ field holds the actual length, while the
1806 field gives the size of the allocated array (not of interest to you,
1807 in most cases). The character array need not be null terminated.
1809 To make things a little easier, an alternative is given for string
1810 types that are not expected to contain embedded NULL characters (eg.
1814 int odr_cstring(ODR o, char **p, int optional, const char *name);
1817 Which encoded or decodes between OCTETSTRING representations and
1818 null-terminates C strings.
1820 Functions are provided for the derived string types, eg:
1823 int odr_visiblestring(ODR o, char **p, int optional, const char *name);
1830 int odr_bitstring(ODR o, Odr_bitmask **p, int optional, const char *name);
1833 The opaque type <tt/Odr_bitmask/ is only suitable for holding relatively brief bit
1834 strings, eg. for options fields, etc. The constant
1835 <tt/ODR_BITMASK_SIZE/
1836 multiplied by 8 gives the maximum possible number of bits.
1838 A set of macros are provided for manipulating the
1843 void ODR_MASK_ZERO(Odr_bitmask *b);
1845 void ODR_MASK_SET(Odr_bitmask *b, int bitno);
1847 void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno);
1849 int ODR_MASK_GET(Odr_bitmask *b, int bitno);
1852 The functions are modelled after the manipulation functions that
1853 accompany the <tt/fd_set/ type used by the <tt/select/(2) call.
1854 <tt/ODR_MASK_ZERO/ should always be called first on a new bitmask, to
1855 initialize the bits to zero.
1857 <sect3>OBJECT IDENTIFIER
1861 int odr_oid(ODR o, Odr_oid **p, int optional, const char *name);
1864 The C OID represenation is simply an array of integers, terminated by
1865 the value -1 (the <tt/Odr_oid/ type is synonymous with the <tt/int/
1866 type). We suggest that you use the OID database module (see section
1867 <ref id="oid" name="Object Identifiers">) to handle object identifiers
1868 in your application.
1870 <sect2>Tagging Primitive Types<label id="tag-prim">
1873 The simplest way of tagging a type is to use the <tt/odr_implicit_tag()/ or
1874 <tt/odr_explicit_tag()/ macros:
1877 int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag, int
1878 optional, const char *name);
1880 int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag,
1881 int optional, const char *name);
1884 To create a type derived from the integer type by implicit tagging, you
1888 MyInt ::= [210] IMPLICIT INTEGER
1891 In the <bf/ODR/ system, this would be written like:
1894 int myInt(ODR o, int **p, int optional, const char *name)
1896 return odr_implicit_tag(o, odr_integer, p,
1897 ODR_CONTEXT, 210, optional, name);
1901 The function <tt/myInt()/ can then be used like any of the primitive
1902 functions provided by ODR. Note that the behavior of
1904 and <tt/odr_implicit()/ macros act exactly the same as the functions they
1905 are applied to - they respond to error conditions, etc, in the same
1906 manner - they simply have three extra parameters. The class parameter may
1907 take one of the values: <tt/ODR_CONTEXT/, <tt/ODR_PRIVATE/,
1908 <tt/ODR_UNIVERSAL/, or
1909 <tt/ODR_APPLICATION/.
1911 <sect2>Constructed Types
1914 Constructed types are created by combining primitive types. The
1916 system only implements the SEQUENCE and SEQUENCE OF constructions
1917 (although adding the rest of the container types should be simple
1918 enough, if the need arises).
1920 For implementing SEQUENCEs, the functions
1923 int odr_sequence_begin(ODR o, void *p, int size, const char *name);
1924 int odr_sequence_end(ODR o);
1929 The <tt/odr_sequence_begin()/ function should be called in the beginning of a
1930 function that implements a SEQUENCE type. Its parameters are the
1932 stream, a pointer (to a pointer to the type you're implementing), and
1933 the <tt/size/ of the type (typically a C structure). On encoding, it
1934 returns 1 if *<tt/p/ is a null pointer. The <tt/size/ parameter is ignored. On
1935 decoding, it returns 1 if the type is found in the data stream.
1937 bytes of memory are allocated, and *<tt/p/ is set to point to this space.
1938 <tt/odr_sequence_end()/ is called at the end of the complex function. Assume
1939 that a type is defined like this:
1942 MySequence ::= SEQUENCE {
1944 boolval BOOLEAN OPTIONAL }
1947 The corresponding ODR encoder/decoder function and the associated data
1948 structures could be written like this:
1951 typedef struct MySequence
1957 int mySequence(ODR o, MySequence **p, int optional, const char *name)
1959 if (odr_sequence_begin(o, p, sizeof(**p), name) == 0)
1960 return optional &ero;&ero; odr_ok(o);
1962 odr_integer(o, &ero;(*p)->intval, 0, "intval") &ero;&ero;
1963 odr_bool(o, &ero;(*p)->boolval, 1, "boolval") &ero;&ero;
1964 odr_sequence_end(o);
1968 Note the 1 in the call to <tt/odr_bool()/, to mark that the sequence
1969 member is optional. If either of the member types had been tagged, the
1970 macros <tt/odr_implicit()/ or <tt/odr_explicit()/ could have been used.
1972 function can be used exactly like the standard functions provided with
1973 <bf/ODR/. It will encode, decode or pretty-print a data value of the
1974 <tt/MySequence/ type. We like to name types with an initial capital, as
1975 done in ASN.1 definitions, and to name the corresponding function with
1976 the first character of the name in lower case. You could, of course,
1977 name your structures, types, and functions any way you please - as
1978 long as you're consistent, and your code is easily readable.
1980 just that - a predicate that returns the state of the stream. It is
1981 used to ensure that the behaviour of the new type is compatible with
1982 the interface of the primitive types.
1984 <sect2>Tagging Constructed Types
1988 NOTE: See section <ref id="tag-prim" name="Tagging Primitive types">
1989 for information on how to tag the primitive types, as well as types
1990 that are already defined.
1993 <sect3>Implicit Tagging
1996 Assume the type above had been defined as
1999 MySequence ::= [10] IMPLICIT SEQUENCE {
2001 boolval BOOLEAN OPTIONAL }
2004 You would implement this in <bf/ODR/ by calling the function
2007 int odr_implicit_settag(ODR o, int class, int tag);
2010 which overrides the tag of the type immediately following it. The
2011 macro <tt/odr_implicit()/ works by calling <tt/odr_implicit_settag()/
2013 before calling the function pointer argument. Your type function could
2017 int mySequence(ODR o, MySequence **p, int optional, const char *name)
2019 if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 ||
2020 odr_sequence_begin(o, p, sizeof(**p), name) == 0)
2021 return optional &ero;&ero; odr_ok(o);
2023 odr_integer(o, &ero;(*p)->intval, 0, "intval") &ero;&ero;
2024 odr_bool(o, &ero;(*p)->boolval, 1, "boolval") &ero;&ero;
2025 odr_sequence_end(o);
2029 The definition of the structure <tt/MySequence/ would be the same.
2031 <sect3>Explicit Tagging
2034 Explicit tagging of constructed types is a little more complicated,
2035 since you are in effect adding a level of construction to the data.
2037 Assume the definition:
2040 MySequence ::= [10] IMPLICIT SEQUENCE {
2042 boolval BOOLEAN OPTIONAL }
2045 Since the new type has an extra level of construction, two new functions
2046 are needed to encapsulate the base type:
2049 int odr_constructed_begin(ODR o, void *p, int class, int tag,
2052 int odr_constructed_end(ODR o);
2055 Assume that the IMPLICIT in the type definition above were replaced
2056 with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which
2057 would be equivalent). The structure definition would look the same,
2058 but the function would look like this:
2061 int mySequence(ODR o, MySequence **p, int optional, const char *name)
2063 if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0)
2064 return optional &ero;&ero; odr_ok(o);
2065 if (o->direction == ODR_DECODE)
2066 *p = odr_malloc(o, sizeof(**p));
2067 if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0)
2069 *p = 0; /* this is almost certainly a protocol error */
2073 odr_integer(o, &ero;(*p)->intval, 0, "intval") &ero;&ero;
2074 odr_bool(o, &ero;(*p)->boolval, 1, "boolval") &ero;&ero;
2075 odr_sequence_end(o) &ero;&ero;
2076 odr_constructed_end(o);
2080 Notice that the interface here gets kind of nasty. The reason is
2081 simple: Explicitly tagged, constructed types are fairly rare in
2082 the protocols that we care about, so the
2083 aesthetic annoyance (not to mention the dangers of a cluttered
2084 interface) is less than the time that would be required to develop a
2085 better interface. Nevertheless, it is far from satisfying, and it's a
2086 point that will be worked on in the future. One option for you would
2087 be to simply apply the <tt/odr_explicit()/ macro to the first function,
2089 have to worry about <tt/odr_constructed_*/ yourself. Incidentally, as you
2090 might have guessed, the <tt/odr_sequence_/ functions are themselves
2091 implemented using the <tt/odr_constructed_/ functions.
2096 To handle sequences (arrays) of a apecific type, the function
2099 int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional),
2100 void *p, int *num, const char *name);
2103 The <tt/fun/ parameter is a pointer to the decoder/encoder
2104 function of the type. <tt/p/ is a pointer to an array of pointers to your
2105 type. <tt/num/ is the number of elements in the array.
2110 MyArray ::= SEQUENCE OF INTEGER
2113 The C representation might be
2116 typedef struct MyArray
2123 And the function might look like
2126 int myArray(ODR o, MyArray **p, int optional, const char *name)
2128 if (o->direction == ODR_DECODE)
2129 *p = odr_malloc(o, sizeof(**p));
2130 if (odr_sequence_of(o, odr_integer, &ero;(*p)->elements,
2131 &ero;(*p)->num_elements, name))
2134 return optional &ero;&ero; odr_ok(o);
2141 The choice type is used fairly often in some ASN.1 definitions, so
2142 some work has gone into streamlining its interface.
2144 CHOICE types are handled by the function:
2147 int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp,
2151 The <tt/arm/ array is used to describe each of the possible types that the
2152 CHOICE type may assume. Internally in your application, the CHOICE
2153 type is represented as a discriminated union. That is, a C union
2154 accompanied by an integer (or enum) identifying the active 'arm' of
2155 the union. <tt/whichp/ is a pointer to the union discriminator. When
2156 encoding, it is examined to determine the current type. When decoding,
2157 it is set to reference the type that was found in the input stream.
2159 The Odr_arm type is defined thus:
2162 typedef struct odr_arm
2173 The interpretation of the fields are:
2176 <tag/tagmode/Either <tt/ODR_IMPLICIT/, <tt/ODR_EXPLICIT/, or <tt/ODR_NONE/
2179 <tag/class, tag/The class and tag of the type (-1 if no tagging is
2181 <tag/which/The value of the discriminator that corresponds to
2182 this CHOICE element. Typically, it will be a #defined constant, or
2184 <tag/fun/A pointer to a function that implements the type of
2185 the CHOICE member. It may be either a standard <bf/ODR/ type or a type
2186 defined by yourself.
2187 <tag/name/Name of tag.
2190 A handy way to prepare the array for use by the <tt/odr_choice()/ function
2192 define it as a static, initialized array in the beginning of your
2193 decoding/encoding function. Assume the type definition:
2196 MyChoice ::= CHOICE {
2198 tagged [99] IMPLICIT INTEGER,
2203 Your C type might look like
2206 typedef struct MyChoice
2223 And your function could look like this:
2226 int myChoice(ODR o, MyChoice **p, int optional, const char *name)
2228 static Odr_arm arm[] =
2230 {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"},
2231 {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer,
2233 {-1, -1, -1, MyChoice_other, odr_boolean, "other"},
2237 if (o->direction == ODR_DECODE)
2238 *p = odr_malloc(o, sizeof(**p);
2240 return optional &ero;&ero; odr_ok(o);
2242 if (odr_choice(o, arm, &ero;(*p)->u, &ero;(*p)->which), name)
2245 return optional &ero;&ero; odr_ok(o);
2249 In some cases (say, a non-optional choice which is a member of a sequence),
2250 you can &dquot;embed&dquot; the union and its discriminator in the structure
2251 belonging to the enclosing type, and you won't need to fiddle with
2252 memory allocation to create a separate structure to wrap the
2253 discriminator and union.
2255 The corresponding function is somewhat nicer in the Sun XDR interface.
2256 Most of the complexity of this interface comes from the possibility of
2257 declaring sequence elements (including CHOICEs) optional.
2259 The ASN.1 specifictions naturally requires that each member of a
2260 CHOICE have a distinct tag, so they can be told apart on decoding.
2261 Sometimes it can be useful to define a CHOICE that has multiple types
2262 that share the same tag. You'll need some other mechanism, perhaps
2263 keyed to the context of the CHOICE type. In effect, we would like to
2264 introduce a level of context-sensitiveness to our ASN.1 specification.
2265 When encoding an internal representation, we have no problem, as long
2266 as each CHOICE member has a distinct discriminator value. For
2267 decoding, we need a way to tell the choice function to look for a
2268 specific arm of the table. The function
2271 void odr_choice_bias(ODR o, int what);
2274 provides this functionality. When called, it leaves a notice for the
2275 next call to <tt/odr_choice()/ to be called on the decoding
2276 stream <tt/o/ that only the <tt/arm/ entry with a <tt/which/ field
2277 equal to <tt/what/ should be tried.
2279 The most important application (perhaps the only one, really) is in
2280 the definition of application-specific EXTERNAL encoders/decoders
2281 which will automatically decode an ANY member given the direct or
2287 The protocol modules are suffering somewhat from a lack of diagnostic
2288 tools at the moment. Specifically ways to pretty-print PDUs that
2289 aren't recognized by the system. We'll include something to this end
2290 in a not-too-distant release. In the meantime, what we do when we get
2291 packages we don't understand is to compile the ODR module with
2292 <tt/ODR_DEBUG/ defined. This causes the module to dump tracing
2293 information as it processes data units. With this output and the
2294 protocol specification (Z39.50), it is generally fairly easy to see
2297 <sect>The COMSTACK Module<label id="comstack">
2303 subsystem provides a transparent interface to different types of transport
2304 stacks for the exchange of BER-encoded data. At present, the
2305 RFC1729 method (BER over TCP/IP), and Peter Furniss' XTImOSI
2306 stack are supported, but others may be added in time. The philosophy of the
2307 module is to provide a simple interface by hiding unused options and
2308 facilities of the underlying libraries. This is always done at the risk
2309 of losing generality, and it may prove that the interface will need
2312 The interface is implemented in such a fashion that only the
2313 sub-layers constructed to the transport methods that you wish to
2314 use in your application are linked in.
2316 You will note that even though simplicity was a goal in the design,
2317 the interface is still orders of magnitudes more complex than the
2318 transport systems found in many other packages. One reason is that
2319 the interface needs to support the somewhat different requirements of
2320 the different lower-layer communications stacks; another important reason is
2321 that the interface seeks to provide a more or less industrial-strength
2322 approach to asynchronous event-handling. When no function is allowed
2323 to block, things get more complex - particularly on the server
2324 side. We urge you to have a look at the demonstration client and server
2325 provided with the package. They are meant to be easily readable and
2326 instructive, while still being at least moderately useful.
2328 <sect1>Common Functions
2330 <sect2>Managing Endpoints
2334 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
2337 Creates an instance of the protocol stack - a communications endpoint.
2340 parameter determines the mode of communication. At present, the
2345 are recognized. The function returns a null-pointer if a system error
2346 occurs. The <tt/blocking/ parameter should be one if you wish the
2347 association to operate in blocking mode, zero otherwise. The
2348 <tt/protocol/ field should be one of <tt/PROTO_SR/ or <tt/PROTO_Z3950/.
2351 int cs_close(COMSTACK handle);
2354 Closes the connection (as elegantly as the lower layers will permit),
2355 and releases the resouces pointed to by the
2359 should not be referenced again after this call.
2363 We really need a soft disconnect, don't we?
2366 <sect2>Data Exchange
2370 int cs_put(COMSTACK handle, char *buf, int len);
2375 down the wire. In blocking mode, this function will return only when a
2376 full buffer has been written, or an error has occurred. In nonblocking
2377 mode, it's possible that the function will be unable to send the full
2378 buffer at once, which will be indicated by a return value of 1. The
2379 function will keep track of the number of octets already written; you
2380 should call it repeatedly with the same values of <tt/buf/ and
2381 <tt/len/, until the buffer has been transmitted. When a full buffer
2382 has been sent, the function will return 0 for success. -1 indicates
2383 an error condition (see below).
2386 int cs_get(COMSTACK handle, char **buf, int *size);
2389 Receives a PDU from the peer. Returns the number of bytes
2390 read. In nonblocking mode, it is possible that not all of the packet can be
2391 read at once. In this case, the function returns 1. To simplify the
2392 interface, the function is
2393 responsible for managing the size of the buffer. It will be reallocated
2394 if necessary to contain large packages, and will sometimes be moved
2395 around internally by the subsystem when partial packages are read. Before
2398 for the fist time, the buffer can be initialized to the null pointer,
2399 and the length should also be set to 0 - cs_get will perform a
2401 on the buffer for you. When a full buffer has been read, the size of
2402 the package is returned (which will always be greater than 1). -1
2403 indicates an error condition.
2410 int cs_more(COMSTACK handle);
2413 The <tt/cs_more()/ function should be used in conjunction with
2417 The <tt/cs_get()/ function will sometimes (notably in the TCP/IP mode) read
2418 more than a single protocol package off the network. When this
2419 happens, the extra package is stored by the subsystem. After calling
2420 <tt/cs_get()/, and before waiting for more input,
2421 You should always call
2423 to check if there's a full protocol package already read. If
2427 can be used to immediately fetch the new package. For the
2429 subsystem, the function should always return 0, but if you want your
2430 stuff to be protocol independent, you should use it.
2433 NOTE: The <tt/cs_more()/
2434 function is required because the
2436 does not provide a way of separating individual PDUs, short of
2437 partially decoding the BER. Some other implementations will carefully
2438 nibble at the packet by calling
2440 several times. This was felt to be too inefficient (or at least
2441 clumsy) - hence the call for this extra function.
2445 int cs_look(COMSTACK handle);
2448 This function is useful when you're operating in nonblocking
2451 tells you there's something happening on the line. It returns one of
2452 the following values:
2455 <tag/CS_NONE/No event is pending. The data found on the line was
2456 not a complete package.
2457 <tag/CS_CONNECT/A response to your connect request has been received. Call
2459 to process the event and to finalize the connection establishment.
2460 <tag/CS_DISCON/The other side has closed the connection (or maybe
2462 request - but do we care? Maybe later). Call
2463 <tt/cs_close/ To close your end of the association as well.
2464 <tag/CS_LISTEN/A connect request has been received. Call
2466 to process the event.
2467 <tag/CS_DATA/There's data to be found on the line. Call
2474 You should be aware that even if
2476 tells you that there's an event event pending, the corresponding
2477 function may still return and tell you there was nothing to be found.
2478 This means that only part of a package was available for reading. The
2479 same event will show up again, when more data has arrived.
2483 int cs_fileno(COMSTACK h);
2486 Returns the file descriptor of the association. Use this when
2487 file-level operations on the endpoint are required (<tt/select/(2)
2488 operations, specifically).
2494 int cs_connect(COMSTACK handle, void *address);
2497 Initiate a connection with the target at <tt/address/ (more on
2498 addresses below). The function will return 0 on success, and 1 if
2499 the operation does not complete immediately (this will only
2500 happen on a nonblocking endpoint). In this case, use
2501 <tt/cs_rcvconnect/ to complete the operation, when <tt/select/(2)
2502 reports input pending on the association.
2505 int cs_rcvconnect(COMSTACK handle);
2508 Complete a connect operation initiated by <tt/cs_connect()/. It will
2509 return 0 on success; 1 if the operation has not yet completed (in this
2510 case, call the function again later); -1 if an error has occured.
2515 To establish a server under the <tt/inetd/ server, you can use
2518 COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
2522 The <it/socket/ parameter is an established socket (when your
2523 application is invoked from <tt/inetd/, the socket will typically be
2524 0. The following parameters are identical to the ones for
2528 int cs_bind(COMSTACK handle, void *address, int mode)
2531 Binds a local address to the endpoint. Read about addresses below. The
2532 <tt/mode/ parameter should be either <tt/CS_CLIENT/ or <tt/CS_SERVER/.
2535 int cs_listen(COMSTACK handle, char *addr, int *addrlen);
2538 Call this to process incoming events on an endpoint that has been
2539 bound in listening mode. It will return 0 to indicate that the connect
2540 request has been received, 1 to signal a partial reception, and -1 to
2541 indicate an error condition.
2544 COMSTACK cs_accept(COMSTACK handle);
2547 This finalises the server-side association establishment, after
2548 cs_listen has completed successfully. It returns a new connection
2549 endpoint, which represents the new association. The application will
2550 typically wish to fork off a process to handle the association at this
2551 point, and continue listen for new connections on the old <tt/handle/.
2553 You can use the call
2556 char *cs_addrstr(COMSTACK);
2559 on an established connection to retrieve the hostname of the remote host.
2561 <it>NOTE: You may need to use this function with some care if your
2562 name server service is slow or unreliable</it>
2567 The low-level format of the addresses are different depending on the
2568 mode of communication you have chosen. A function is provided by each
2569 of the lower layers to map a user-friendly string-form address to the
2570 binary form required by the lower layers.
2573 struct sockaddr_in *tcpip_strtoaddr(char *str);
2575 struct netbuf *mosi_strtoaddr(char *str);
2578 The format for TCP/IP addresses is straightforward:
2581 <host> [ ':' <portnum> ]
2584 The <tt/hostname/ can be either a domain name or an IP address. The
2585 port number, if omitted, defaults to 210.
2587 For OSI, the format is
2590 [ <t-selector> '/' ] <host> [ ':' <port> ]
2593 The transport selector is given as an even number of hex digits.
2595 You'll note that the address format for the OSI mode are just a subset
2596 of full presentation addresses. We use presentation addresses because
2597 xtimosi doesn't, in itself, allow access to the X.500 Directory
2598 service. We use a limited form, because we haven't yet come across an
2599 implementation that used more of the elements of a full p-address. It
2600 is a fairly simple matter to add the rest of the elements to the
2601 address format as needed, however: Xtimosi <it/does/ support the full
2602 P-address structure.
2604 In both transport modes, the special hostname &dquot;@&dquot; is mapped
2605 to any local address (the manifest constant INADDR_ANY). It is used
2606 to establish local listening endpoints in the server role.
2608 When a connection has been established, you can use
2611 char cs_addrstr(COMSTACK h);
2614 to retrieve the host name of the peer system. The function returns a pointer
2615 to a static area, which is overwritten on the next call to the function.
2618 NOTE: We have left the issue of X.500 name-to-address mapping open, for the
2619 moment. It would be a simple matter to provide a table-based mapping,
2620 if desired. Alternately, we could use the X.500 client-function that
2621 is provided with the ISODE (although this would defeat some of the
2622 purpose of using ThinOSI in the first place. We have been told that it
2623 should be within the realm of the possible to implement a lightweight
2624 implementation of the necessary X.500 client capabilities on top of
2625 ThinOSI. This would be the ideal solution, we feel. On the other hand, it
2626 still remains to be seen just what role the Directory will play in a world
2627 populated by ThinOSI and other pragmatic solutions.
2633 All functions return -1 if an error occurs. Typically, the functions
2634 will return 0 on success, but the data exchange functions
2635 (<tt/cs_get/, <tt/cs_put/, <tt/cs_more/)
2636 follow special rules. Consult their descriptions.
2638 When a function (including the data exchange functions) reports an
2642 to determine the cause of the
2643 problem. The function
2646 void cs_perror(COMSTACK handle char *message);
2653 argument, along with a system message, to
2655 Use the character array
2658 extern const char *cs_errlist[];
2661 to get hold of the message, if you want to process it differently.
2665 const char *cs_stackerr(COMSTACK handle);
2668 Returns an error message from the lower layer, if one has been
2671 <sect1>Enabling OSI Communication
2673 <sect2>Installing Xtimosi
2675 Although you will have to download Peter Furniss' XTI/mOSI
2676 implementation for yourself, we've tried to make the integration as
2679 The latest version of xtimosi will generally be under
2682 ftp://pluto.ulcc.ac.uk/ulcc/thinosi/xtimosi/
2685 When you have downloaded and unpacked the archive, it will (we assume)
2686 have created a directory called <tt/xtimosi/. We suggest that you place
2687 this directory <it/in the same directory/ where you unpacked the
2689 distribution. This way, you shouldn't have to fiddle with the
2690 makefiles of <tt/YAZ/ beyond uncommenting a few lines.
2692 Go to <tt>xtimosi/src</tt>, and type &dquot;<tt/make libmosi.a/&dquot;.
2693 This should generally
2694 create the library, ready to use.
2699 The currently available release of xtimosi has some inherent
2700 problems that make it disfunction on certain platforms - eg. the
2701 Digital OSF/1 workstations. It is supposedly primarily a
2702 compiler problem, and we hope to see a release that is generally
2703 portable. While we can't guarantee that it can be brought to work
2704 on your platform, we'll be happy to talk to you about problems
2705 that you might see, and relay information to the author of the
2706 software. There are some signs that the <bf/gcc/ compiler is more
2707 likely to produce a fully functional library, but this hasn't
2708 been verified (we think that the problem is limited to the use
2709 of hexadecimal escape-codes used in strings, which are silently
2710 ignored by some compilers).
2714 A problem has been encountered in the communication with
2715 ISODE-based applications. If the ISODE presentation-user calls
2716 <tt/PReadRequest()/ with a timeout value different from <tt/OK/ or <tt/NOTOK/,
2717 he will get an immediate TIMEOUT abort when receiving large (>2041
2718 bytes, which is the SPDU-size that the ISODE likes to work with) packages
2719 from an xtimosi-based implementation (probably most
2720 other implementations as well, in fact). It seems to be a flaw in the
2721 ISODE API, and the workaround (for ISODE users) is to either not
2722 use an explicit timeout (switching to either blocking or
2723 nonblocking mode), or to check that the timer really has expired
2724 before closing the connection.
2727 The next step in the installation is to modify the makefile in the toplevel
2729 directory. The place to change is in the top of the file, and is
2730 clearly marked with a comment.
2732 Now run <tt/make/ in the <bf/YAZ/ toplevel directory (do a
2733 &dquot;<tt/make clean/&dquot; first, if the
2734 system has been previously made without OSI support). Use the <bf/YAZ/
2736 and <bf/client/ demo programs to verify that OSI communication works OK. Then,
2737 you can go ahead and try to talk to other implementations.
2740 NOTE: Our interoperability experience is limited to version
2741 7 of the Nordic SR-Nett package, which has had several
2742 protocol errors fixed from the earlier releases. If you have
2743 problems or successes in interoperating with other
2744 implementations, we'd be glad to hear about it, or to help
2745 you make things work, as our resources allow.
2748 If you write your own applications based on <bf/YAZ/, and you wish to
2749 include OSI support, the procedure is equally simple. You should
2750 include the <tt/xmosi.h/ header file in addition to <tt/comstack.h/.
2752 will define the manifest constant <tt/mosi_type/, which you should pass
2753 to the <tt/cs_create()/ function. In
2754 addition, you should use the function <tt/mosi_strtoaddr()/ rather than
2755 <tt/tcpip_strtoaddr()/ when you need to prepare an address.
2757 When you link your application, you should include (after the
2759 library) the <tt/libmosi.a/ library, and the <tt/librfc.a/ library
2761 <bf/YAZ/ (for OSI transport).
2763 As always, it can be very useful, if not essential, to have a look at the
2764 example applications
2765 to see how things are done.
2767 <sect2>OSI Transport
2770 Xtimosi requires an implementation of the OSI transport service under
2771 the X/OPEN XTI API. We provide an implementation of the RFC1006
2772 encapsulation of OSI/TP0 in TCP/IP (through the Berkeley Sockets API),
2773 as an independent part of <bf/YAZ/ (it's found under the <tt/rfc1006/
2774 directory). If you have access to an OSI transport provider under XTI,
2775 you should be able to make that work too, although it may require
2776 tinkering with the <tt/mosi_strtoaddr()/ function.
2778 <sect2>Presentation Context Management
2781 To simplify the implementation, we use Peter Furniss' alternative (PRF)
2783 for the Control of the presentation negotiation phase. This format
2784 is enabled by default when you
2787 The current version of <bf/YAZ/ does <it/not/ support presentation-layer
2788 negotiation of response record formats. The primary reason is that we
2789 have had access to no other SR or Z39.50 implementations over OSI that
2791 method. Secondarily, we believe that the EXPLAIN facility is a superior
2792 mechanism for relaying target capabilities in this respect. This is not to
2793 say that we have no intentions of supporting presentation context
2794 negotiation - we have just hitherto given it a lower priority than other
2795 aspects of the protocol.
2797 One thing is certain: The addition of this capability to <bf/YAZ/ should
2798 have only a minimal impact on existing applications, and on the
2799 interface to the software in general. Most likely, we will add an extra
2800 layer of interface to the processing of EXPLAIN records, which will
2801 convert back and forth between <tt/oident/ records (see section
2802 <ref id="oid" name="Object Identifiers">) and direct or indirect
2803 references, given the current association setup. Implementations based
2804 on any of the higher-level interfaces will most likely not have to be
2807 <sect1>Summary and Synopsis
2811 #include <comstack.h>
2813 #include <tcpip.h> /* this is for TCP/IP support */
2814 #include <xmosi.h> /* and this is for mOSI support */
2816 COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
2818 COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
2821 int cs_bind(COMSTACK handle, int mode);
2823 int cs_connect(COMSTACK handle, void *address);
2825 int cs_rcvconnect(COMSTACK handle);
2827 int cs_listen(COMSTACK handle);
2829 COMSTACK cs_accept(COMSTACK handle);
2831 int cs_put(COMSTACK handle, char *buf, int len);
2833 int cs_get(COMSTACK handle, char **buf, int *size);
2835 int cs_more(COMSTACK handle);
2837 int cs_close(COMSTACK handle);
2839 int cs_look(COMSTACK handle);
2841 struct sockaddr_in *tcpip_strtoaddr(char *str);
2843 struct netbuf *mosi_strtoaddr(char *str);
2845 extern int cs_errno;
2847 void cs_perror(COMSTACK handle char *message);
2849 const char *cs_stackerr(COMSTACK handle);
2851 extern const char *cs_errlist[];
2854 <sect>Making an IR Interface for Your Database with YAZ<label id="server">
2860 NOTE: If you aren't into documentation, a good way to learn how the
2861 backend interface works is to look at the backend.h file. Then,
2862 look at the small dummy-server in server/ztest.c. Finally, you can
2863 have a look at the seshigh.c file, which is where most of the
2864 logic of the frontend server is located. The backend.h file also
2865 makes a good reference, once you've chewed your way through
2866 the prose of this file.
2869 If you have a database system that you would like to make available by
2870 means of Z39.50/SR, <bf/YAZ/ basically offers your two options. You
2871 can use the APIs provided by the <bf/ASN/, <bf/ODR/, and <bf/COMSTACK/
2873 create and decode PDUs, and exchange them with a client. Using this
2874 low-level interface gives you access to all fields and options of the
2875 protocol, and you can construct your server as close to your existing
2876 database as you like. It is also a fairly involved process, requiring
2877 you to set up an event-handling mechanism, protocol state machine,
2878 etc. To simplify server implementation, we have implemented a compact
2879 and simple, but reasonably full-functioned server-frontend that will
2880 handle most of the protocol mechanics, while leaving you to
2881 concentrate on your database interface.
2884 NOTE: The backend interface was designed in anticipation of a specific
2885 integration task, while still attempting to achieve some degree of
2886 generality. We realise fully that there are points where the
2887 interface can be improved significantly. If you have specific
2888 functions or parameters that you think could be useful, send us a
2889 mail (or better, sign on to the mailing list referred to in the
2890 toplevel README file). We will try to fit good suggestions into future
2891 releases, to the extent that it can be done without requiring
2892 too many structural changes in existing applications.
2895 <sect1>The Database Frontend
2898 We refer to this software as a generic database frontend. Your
2899 database system is the <it/backend database/, and the interface between
2900 the two is called the <it/backend API/. The backend API consists of a
2901 small number of function prototypes and structure definitions. You are
2902 required to provide the <bf/main()/ routine for the server (which can be
2903 quite simple), as well as functions to match each of the prototypes.
2904 The interface functions that you write can use any mechanism you like
2905 to communicate with your database system: You might link the whole
2906 thing together with your database application and access it by
2907 function calls; you might use IPC to talk to a database server
2908 somewhere; or you might link with third-party software that handles
2909 the communication for you (like a commercial database client library).
2910 At any rate, the functions will perform the tasks of:
2913 <item>Initialization.
2915 <item>Fetching records.
2916 <item>Scanning the database index (if you wish to implement SCAN).
2919 (more functions will be added in time to support as much of
2920 Z39.50-1995 as possible).
2922 Because the model where pipes or sockets are used to access the backend
2923 database is a fairly common one, we have added a mechanism that allows this
2924 communication to take place asynchronously. In this mode, the frontend
2925 server doesn't have to block while the backend database is processing
2926 a request, but can wait for additional PDUs from the client.
2928 <sect1>The Backend API
2931 The headers files that you need to use the interface are in the
2932 include/ directory. They are called <tt/statserv.h/ and <tt/backend.h/. They
2933 will include other files from the <tt/include/ directory, so you'll
2934 probably want to use the -I option of your compiler to tell it where
2935 to find the files. When you run <tt/make/ in the toplevel <bf/YAZ/ directory,
2936 everything you need to create your server is put the lib/libyaz.a
2937 library. If you want OSI as well, you'll also need to link in the
2938 <tt/libmosi.a/ library from the xtimosi distribution (see the mosi.txt
2939 file), a well as the <tt>lib/librfc.a</tt> library (to provide OSI transport
2942 <sect1>Your main() Routine
2945 As mentioned, your <bf/main()/ routine can be quite brief. If you want to
2946 initialize global parameters, or read global configuration tables,
2947 this is the place to do it. At the end of the routine, you should call
2950 <tscreen><verb>int statserv_main(int argc, char **argv);</verb></tscreen>
2952 <bf/Statserv_main/ will establish listening sockets according to the
2953 parameters given. When connection requests are received, the event
2954 handler will typically <bf/fork()/ to handle the new request. If you do use
2955 global variables, you should be aware, then, that these cannot be
2956 shared between associations, unless you explicitly disallow forking by
2957 command line parameters (we advise against this for any purposes
2958 except debugging, as a crash or hang in the server process will affect
2959 all users currently signed on to the server).
2961 The server provides a mechanism for controlling some of its behavior
2962 without using command-line options. The function
2965 statserv_options_block *statserv_getcontrol(void);
2968 Will return a pointer to a <tt/struct statserv_options_block/ describing
2969 the current default settings of the server. The structure contains
2973 <tag/int dynamic/A boolean value, which determines whether the server
2974 will fork on each incoming request (TRUE), or not (FALSE). Default is
2976 <tag/int loglevel/Set this by ORing the constants defined in include/log.h.
2977 <tag/char logfile[ODR_MAXNAME+1]/File for diagnostic output
2978 (&dquot;&dquot;: stderr).
2979 <tag/char apdufile[ODR_MAXNAME+1]/Name of file for logging incoming and
2980 outgoing APDUs (&dquot;&dquot;: don't log APDUs, &dquot;-&dquot;: <tt/stderr/).
2981 <tag/char default_listen[1024]/Same form as the command-line
2982 specification of listener address. &dquot;&dquot;: no default listener address.
2983 Default is to listen at &dquot;tcp:@:9999&dquot;. You can only
2984 specify one default listener address in this fashion.
2985 <tag/enum oid_proto default_proto;/Either <tt/PROTO_SR/ or <tt/PROTO_Z3950/.
2986 Default is <tt/PROTO_Z39_50/.
2987 <tag/int idle_timeout;/Maximum session idletime, in minutes. Zero indicates
2988 no (infinite) timeout. Default is 120 minutes.
2989 <tag/int maxrecordsize;/Maximum permissible record (message) size. Default
2990 is 1Mb. This amount of memory will only be allocated if a client requests a
2991 very large amount of records in one operation (or a big record). Set it
2993 if you are worried about resource consumption on your host system.
2994 <tag/char configname[ODR_MAXNAME+1]/Passed to the backend when a
2995 new connection is received.
2996 <tag/char setuid[ODR_MAXNAME+1]/Set user id to the user specified,
2997 after binding the listener addresses.
3000 The pointer returned by <tt/statserv_getcontrol/ points to a static area.
3001 You are allowed to change the contents of the structure, but the
3002 changes will not take effect before you call
3005 void statserv_setcontrol(statserv_options_block *block);
3008 Note that you should generally update this structure <it/before/ calling
3009 <tt/statserv_main()/.
3011 <sect1>The Backend Functions
3014 For each service of the protocol, the backend interface declares one or
3015 two functions. You are required to provide implementations of the
3016 functions representing the services that you wish to implement.
3018 <tscreen><verb>bend_initresult *bend_init(bend_initrequest *r);</verb></tscreen>
3020 This function is called once for each new connection request, after
3021 a new process has been forked, and an initRequest has been received
3022 from the client. The parameter and result structures are defined as
3026 typedef struct bend_initrequest
3031 typedef struct bend_initresult
3033 int errcode; /* 0==OK */
3034 char *errstring; /* system error string or NULL */
3035 void *handle; /* private handle to the backend module */
3040 The <tt/configname/ of <tt/bend_initrequest/ is currently always set to
3041 &dquot;default-config&dquot;. We haven't had use for putting anything special in
3042 the initrequest yet, but something might go there if the need arises
3043 (account/password info would be obvious).
3045 In general, the server frontend expects that the <tt/bend_*result/
3046 pointer that you return is valid at least until the next call to a
3047 <tt/bend_* function/. This applies to all of the functions described
3048 herein. The parameter structure passed to you in the call belongs to
3049 the server frontend, and you should not make assumptions about its
3050 contents after the current function call has completed. In other
3051 words, if you want to retain any of the contents of a request
3052 structure, you should copy them.
3054 The <tt/errcode/ should be zero if the initialization of the backend went
3055 well. Any other value will be interpreted as an error. The
3056 <tt/errstring/ isn't used in the current version, but one option
3057 would be to stick it
3058 in the initResponse as a VisibleString. The <tt/handle/ is the most
3059 important parameter. It should be set to some value that uniquely
3060 identifies the current session to the backend implementation. It is
3061 used by the frontend server in any future calls to a backend function.
3062 The typical use is to set it to point to a dynamically allocated state
3063 structure that is private to your backend module.
3067 bend_searchresult *bend_search(void *handle, bend_searchrequest *r,
3069 bend_searchresult *bend_searchresponse(void *handle);
3071 typedef struct bend_searchrequest
3073 char *setname; /* name to give to this set */
3074 int replace_set; /* replace set, if it already exists */
3075 int num_bases; /* number of databases in list */
3076 char **basenames; /* databases to search */
3077 Z_Query *query; /* query structure */
3078 } bend_searchrequest;
3080 typedef struct bend_searchresult
3082 int hits; /* number of hits */
3083 int errcode; /* 0==OK */
3084 char *errstring; /* system error string or NULL */
3085 } bend_searchresult;
3089 The first thing to notice about the search request interface (as well
3090 as all of the following requests), is that it consists of two separate
3091 functions. The idea is to provide a simple facility for
3092 asynchronous communication with the backend server. When a
3093 searchrequest comes in, the server frontend will fill out the
3094 <tt/bend_searchrequest/ tructure, and call the <tt/bend_search
3095 function/. The <tt/fd/
3096 argument will point to an integer variable. If you are able to do
3097 asynchronous I/O with your database server, you should set *<tt/fd/ to the
3098 file descriptor you use for the communication, and return a null
3099 pointer. The server frontend will then <tt/select()/ on the *<tt/fd/,
3101 <tt/bend_searchresult/ when it sees that data is available. If you don't
3102 support asynchronous I/O, you should return a pointer to the
3103 <tt/bend_searchresult/ immediately, and leave *<tt/fd/ untouched. This
3104 construction is common to all of the <tt/bend_/ functions (except
3105 <tt/bend_init/). Note that you can choose to support this facility in none,
3106 any, or all of the <tt/bend_/ functions, and you can respond
3107 differently on each request at run-time. The server frontend will
3110 The <tt/bend_searchrequest/ is a fairly close approximation of a protocol
3111 searchRequest PDU. The <tt/setname/ is the resultSetName from the protocol. You
3112 are required to establish a mapping between the set name and whatever
3113 your backend database likes to use. Similarly, the <tt/replace_set/ is a
3114 boolean value corresponding to the resultSetIndicator field in the
3115 protocol. <tt>Num_bases/basenames</tt> is a length of/array of character
3116 pointers to the database names provided by the client. The <tt/query/ is the
3117 full query structure as defined in the protocol ASN.1 specification.
3118 It can be either of the possible query types, and it's up to you to
3119 determine if you can handle the provided query type. Rather than
3120 reproduce the C interface here, we'll refer you to the structure
3121 definitions in the file <tt>include/proto.h</tt>. If you want to look at the
3122 attributeSetId OID of the RPN query, you can either match it against
3123 your own internal tables, or you can use the <tt/oid_getentbyoid/ function
3124 provided by <bf/YAZ/.
3126 The result structure contains a number of hits, and an
3127 <tt>errcode/errstring</tt> pair. If an error occurs during the search, or if
3128 you're unhappy with the request, you should set the errcode to a value
3129 from the BIB-1 diagnostic set. The value will then be returned to the
3130 user in a nonsurrogate diagnostic record in the response. The
3131 <tt/errstring/, if provided, will go in the addinfo field. Look at the
3132 protocol definition for the defined error codes, and the suggested
3133 uses of the addinfo field.
3137 bend_fetchresult *bend_fetch(void *handle, bend_fetchrequest *r,
3139 bend_fetchresult *bend_fetchresponse(void *handle);
3141 typedef struct bend_fetchrequest
3143 char *setname; /* set name */
3144 int number; /* record number */
3146 } bend_fetchrequest;
3148 typedef struct bend_fetchresult
3150 char *basename; /* name of database that provided record */
3151 int len; /* length of record */
3152 char *record; /* record */
3153 int last_in_set; /* is it? */
3155 int errcode; /* 0==success */
3156 char *errstring; /* system error string or NULL */
3162 NOTE: The <tt/bend_fetchresponse()/ function is not yet supported
3163 in this version of the software. Your implementation of <tt/bend_fetch()/
3164 should always return a pointer to a <tt/bend_fetchresult/.
3167 The frontend server calls <tt/bend_fetch/ when it needs database records to
3168 fulfill a searchRequest or a presentRequest. The <tt/setname/ is simply the
3169 name of the result set that holds the reference to the desired record.
3170 The <tt/number/ is the offset into the set (with 1 being the first record
3171 in the set). The <tt/format/ field is the record format requested by the
3172 client (See section <ref id="oid" name="Object Identifiers">). The value
3173 <tt/VAL_NONE/ indicates that the client did not request a specific format.
3174 The <tt/stream/ argument is an <bf/ODR/ stream which should be used for
3175 allocating space for structured data records. The stream will be reset when
3176 all records have been assembled, and the response package has been transmitted.
3177 For unstructured data, the backend is responsible for maintaining a static
3178 or dynamic buffer for the record between calls.
3180 In the result structure, the <tt/basename/ is the name of the
3181 database that holds the
3182 record. <tt/Len/ is the length of the record returned, in bytes, and
3184 is a pointer to the record. <tt/Last_in_set/ should be nonzero only if the
3185 record returned is the last one in the given result set. <tt/Errcode/ and
3186 <tt/errstring/, if given, will currently be interpreted as a global error
3187 pertaining to the set, and will be returned in a
3188 nonSurrogateDiagnostic.
3190 <it>NOTE: This is silly. Add a flag to say which is which.</it>
3192 If the <tt/len/ field has the value -1, then <tt/record/ is assumed to point
3193 to a constructed data type. The <tt/format/ field will be used to determine
3194 which encoder should be used to serialize the data.
3197 NOTE: If your backend generates structured records, it should use
3198 <tt/odr_malloc()/ on the provided stream for allocating data: This allows
3199 the frontend server to keep track of the record sizes.
3202 The <tt/format/ field is mapped to an object identifier in the direct
3203 reference of the resulting EXTERNAL representation of the record.
3205 <it>NOTE: The current version of <bf/YAZ/ only supports the direct reference
3210 bend_deleteresult *bend_delete(void *handle, bend_deleterequest *r,
3212 bend_deleteresult *bend_deleteresponse(void *handle);
3214 typedef struct bend_deleterequest
3217 } bend_deleterequest;
3219 typedef struct bend_deleteresult
3221 int errcode; /* 0==success */
3222 char *errstring; /* system error string or NULL */
3223 } bend_deleteresult;
3228 NOTE: The &dquot;delete&dquot; function is not yet supported
3229 in this version of the software.
3233 NOTE: The delete set function definition is rather primitive, mostly
3235 have had no practical need for it as of yet. If someone wants
3236 to provide a full delete service, we'd be happy to add the
3237 extra parameters that are required. Are there clients out there
3238 that will actually delete sets they no longer need?
3243 bend_scanresult *bend_scan(void *handle, bend_scanrequest *r,
3245 bend_scanresult *bend_scanresponse(void *handle);
3247 typedef struct bend_scanrequest
3249 int num_bases; /* number of elements in databaselist */
3250 char **basenames; /* databases to search */
3251 Z_AttributesPlusTerm *term;
3252 int term_position; /* desired index of term in result list */
3253 int num_entries; /* number of entries requested */
3256 typedef struct bend_scanresult
3277 NOTE: The <tt/bend_scanresponse()/ function is not yet supported
3278 in this version of the software. Your implementation of <tt/bend_scan()/
3279 should always return a pointer to a <tt/bend_scanresult/.
3282 <sect1>Application Invocation
3285 The finished application has the following
3286 invocation syntax (by way of <tt/statserv_main()/):
3289 appname [-szSu -a apdufile -l logfile -v loglevel]
3290 [listener ...]
3296 <tag/-a/APDU file. Specify a file for dumping PDUs (for diagnostic purposes).
3297 The special name &dquot;-&dquot; sends output to <tt/stderr/.
3299 <tag/-S/Don't fork on connection requests. This is good for debugging, but
3300 not recommended for real operation: Although the server is
3301 asynchronous and non-blocking, it can be nice to keep a software
3302 malfunction (okay then, a crash) from affecting all current users.
3304 <tag/-s/Use the SR protocol.
3306 <tag/-z/Use the Z39.50 protocol (default). These two options complement
3307 eachother. You can use both multiple times on the same command
3308 line, between listener-specifications (see below). This way, you
3309 can set up the server to listen for connections in both protocols
3310 concurrently, on different local ports.
3312 <tag/-l/The logfile.
3314 <tag/-v/The log level. Use a comma-separated list of members of the set
3315 {fatal,debug,warn,log,all,none}.
3316 <tag/-u/Set user ID. Sets the real UID of the server process to that of the
3317 given user. It's useful if you aren't comfortable with having the
3318 server run as root, but you need to start it as such to bind a
3321 <tag/-w/Working directory.
3322 <tag/-i/Use this when running from the <tt/inetd/ server.
3324 <tag/-t/Idle session timeout, in minutes.
3326 <tag/-k/Maximum record size/message size, in kilobytes.
3330 A listener specification consists of a transport mode followed by a
3331 colon (:) followed by a listener address. The transport mode is
3332 either <tt/osi/ or <tt/tcp/.
3334 For TCP, an address has the form
3337 hostname | IP-number [: portnumber]
3340 The port number defaults to 210 (standard Z39.50 port).
3342 For osi, the address form is
3345 [t-selector /] hostname | IP-number [: portnumber]
3348 The transport selector is given as a string of hex digits (with an even
3349 number of digits). The default port number is 102 (RFC1006 port).
3357 osi:0402/dbserver.osiworld.com:3000
3361 In both cases, the special hostname &dquot;@&dquot; is mapped to
3362 the address INADDR_ANY, which causes the server to listen on any local
3363 interface. To start the server listening on the registered ports for
3364 Z39.50 and SR over OSI/RFC1006, and to drop root privileges once the
3365 ports are bound, execute the server like this (from a root shell):
3368 my-server -u daemon tcp:@ -s osi:@
3371 You can replace <tt/daemon/ with another user, eg. your own account, or
3372 a dedicated IR server account. <tt/my-server/ should be the name of your
3373 server application. You can test the procedure with the <tt/ztest/
3376 <sect1>Summary and Synopsis
3380 #include <backend.h>
3382 bend_initresult *bend_init(bend_initrequest *r);
3384 bend_searchresult *bend_search(void *handle, bend_searchrequest *r,
3387 bend_searchresult *bend_searchresponse(void *handle);
3389 bend_fetchresult *bend_fetch(void *handle, bend_fetchrequest *r,
3392 bend_fetchresult *bend_fetchresponse(void *handle);
3394 bend_scanresult *bend_scan(void *handle, bend_scanrequest *r, int *fd);
3396 bend_scanresult *bend_scanresponse(void *handle);
3398 bend_deleteresult *bend_delete(void *handle, bend_deleterequest *r,
3401 bend_deleteresult *bend_deleteresponse(void *handle);
3403 void bend_close(void *handle);
3406 <sect>Future Directions
3409 The software has been
3410 successfully ported to the Mac as well as Windows NT/95 - we'd like to
3411 test those ports better and make sure they work as they should.
3413 We have a new and better version of the frontend server on the drawing
3414 board. Resources and external commitments will govern when we'll be
3415 able to do something real with it. Fetures should include greater
3416 flexibility, greter support for access/resource control, and easy
3417 support for Explain (possibly with Zebra as an extra database engine).
3419 We now support all PDUs of Z39.50-1995. If there is one of the
3420 supporting structures that you need but can't find in the prt*.h
3421 files, send us a note; it may be on its way.
3423 The 'retrieval' module needs to be finalized and documented. We think
3424 it can form a useful resource for people dealing with complex record
3425 structures, but for now, you'll mostly have to chew through the code
3426 yourself to make use of it. Not acceptable.
3428 Other than that, YAZ generally moves in the directions which appear to
3429 make the most people happy (including ourselves, as prime users of the
3430 software). If there's something you'd like to see in here, then drop
3431 us a note and let's see what we can come up with.
3435 <sect1>Index Data Copyright
3438 Copyright © 1995-2000 Index Data.
3440 Permission to use, copy, modify, distribute, and sell this software and
3441 its documentation, in whole or in part, for any purpose, is hereby granted,
3444 1. This copyright and permission notice appear in all copies of the
3445 software and its documentation. Notices of copyright or attribution
3446 which appear at the beginning of any file must remain unchanged.
3448 2. The names of Index Data or the individual authors may not be used to
3449 endorse or promote products derived from this software without specific
3450 prior written permission.
3452 THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND,
3453 EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY
3454 WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
3455 IN NO EVENT SHALL INDEX DATA BE LIABLE FOR ANY SPECIAL, INCIDENTAL,
3456 INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES
3457 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR
3458 NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF
3459 LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
3462 <sect1>Additional Copyright Statements
3465 The optional CCL query language interpreter is covered by the following
3468 Copyright © 1995, the EUROPAGATE consortium (see below).
3470 The EUROPAGATE consortium members are:
3472 University College Dublin
3473 Danmarks Teknologiske Videnscenter
3474 An Chomhairle Leabharlanna
3475 Consejo Superior de Investigaciones Cientificas
3477 Permission to use, copy, modify, distribute, and sell this software and
3478 its documentation, in whole or in part, for any purpose, is hereby granted,
3481 1. This copyright and permission notice appear in all copies of the
3482 software and its documentation. Notices of copyright or attribution
3483 which appear at the beginning of any file must remain unchanged.
3485 2. The names of EUROPAGATE or the project partners may not be used to
3486 endorse or promote products derived from this software without specific
3487 prior written permission.
3489 3. Users of this software (implementors and gateway operators) agree to
3490 inform the EUROPAGATE consortium of their use of the software. This
3491 information will be used to evaluate the EUROPAGATE project and the
3492 software, and to plan further developments. The consortium may use
3493 the information in later publications.
3495 4. Users of this software agree to make their best efforts, when
3496 documenting their use of the software, to acknowledge the EUROPAGATE
3497 consortium, and the role played by the software in their work.
3499 THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND,
3500 EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY
3501 WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
3502 IN NO EVENT SHALL THE EUROPAGATE CONSORTIUM OR ITS MEMBERS BE LIABLE
3503 FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF
3504 ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA
3505 OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND
3506 ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE
3507 USE OR PERFORMANCE OF THIS SOFTWARE.
3509 <sect>About Index Data
3512 Index Data is a consulting and software-development enterprise that
3513 specialises in library and information management systems. Our
3514 interests and expertise span a broad range of related fields, and one
3515 of our primary, long-term objectives is the development of a powerful
3516 information management
3517 system with open network interfaces and hypermedia capabilities.
3519 We make this software available free of charge, on a fairly unrestrictive
3520 license; as a service to the networking community, and to further the
3521 development of quality software for open network communication.
3523 We'll be happy to answer questions about the software, and about ourselves
3529 DK-2200 Copenhagen N
3534 Phone: +45 3536 3672
3536 Email: info@indexdata.dk
3539 The <it>Hacker's Jargon File</it> has the following to say about the
3541 prefix &dquot;YA&dquot; in the name of a software product.
3544 Yet Another. adj. 1. Of your own work: A
3545 humorous allusion often used in titles to acknowledge that the
3546 topic is not original, though the content is. As in &dquot;Yet Another
3547 AI Group&dquot; or &dquot;Yet Another Simulated Annealing Algorithm&dquot;.
3549 others' work: Describes something of which there are already far
projects / yaz-moved-to-github.git / blob