ed7dda4a69714dccb3923b1e91c45c6f64fb1c1d
[yaz-moved-to-github.git] / doc / asn.xml
1 <!-- $Header: /home/cvsroot/yaz/doc/asn.xml,v 1.3 2001-07-19 16:53:02 adam Exp $ -->
2 <chapter><title>The ASN Module</title>
3 <sect1><title>Introduction</title>
4 <para>
5 The &asn; module provides you with a set of C struct definitions for the
6 various PDUs of the protocol, as well as for the complex types
7 appearing within the PDUs. For the primitive data types, the C
8 representation often takes the form of an ordinary C language type,
9 such as <literal>int</literal>. For ASN.1 constructs that have no direct
10 representation in C, such as general octet strings and bit strings,
11 the &odr; module (see section <link linkend="odr">The ODR Module</link>)
12 provides auxiliary definitions.
13 </para>
14 </sect1>
15 <sect1><title>Preparing PDUs</title>
16
17 <para>
18 A structure representing a complex ASN.1 type doesn't in itself contain the
19 members of that type. Instead, the structure contains
20 <emphasis>pointers</emphasis> to the members of the type.
21 This is necessary, in part, to allow a mechanism for specifying which
22 of the optional structure (SEQUENCE) members are present, and which
23 are not. It follows that you will need to somehow provide space for
24 the individual members of the structure, and set the pointers to
25 refer to the members.
26 </para>
27 <para>
28 The conversion routines don't care how you allocate and maintain your
29 C structures - they just follow the pointers that you provide.
30 Depending on the complexity of your application, and your personal
31 taste, there are at least three different approaches that you may take
32 when you allocate the structures.
33 </para>
34
35 <para>
36 You can use static or automatic local variables in the function that
37 prepares the PDU. This is a simple approach, and it provides the most
38 efficient form of memory management. While it works well for flat
39 PDUs like the InitReqest, it will generally not be sufficient for say,
40 the generation of an arbitrarily complex RPN query structure.
41 </para>
42 <para>
43 You can individually create the structure and its members using the
44 <function>malloc(2)</function> function. If you want to ensure that
45 the data is freed when it is no longer needed, you will have to
46 define a function that individually releases each member of a
47 structure before freeing the structure itself.
48 </para>
49 <para>
50 You can use the <function>odr_malloc()</function> function (see section 
51 <link linkend="odr-use">Using ODR</link> for details). When you use
52 <function>odr_malloc()</function>, you can release all of the
53 allocated data in a single operation, independent of any pointers and
54 relations between the data. <function>odr_malloc()</function> is based on a
55 &quot;nibble-memory&quot;
56 scheme, in which large portions of memory are allocated, and then
57 gradually handed out with each call to <function>odr_malloc()</function>.
58 The next time you call <function>odr_reset()</function>, all of the
59 memory allocated since the last call is recycled for future use (actually,
60 it is placed on a free-list).
61 </para>
62 <para>
63 You can combine all of the methods described here. This will often be
64 the most practical approach. For instance, you might use
65 <function>odr_malloc()</function> to allocate an entire structure and
66 some of its elements, while you leave other elements pointing to global
67 or per-session default variables.
68 </para>
69
70 <para>
71 The &asn; module provides an important aid in creating new PDUs. For
72 each of the PDU types (say, <function>Z_InitRequest</function>), a
73 function is provided that allocates and initializes an instance of
74 that PDU type for you. In the case of the InitRequest, the function is
75 simply named <function>zget_InitRequest()</function>, and it sets up
76 reasonable default value for all of the mandatory members. The optional
77 members are generally initialized to null pointers. This last aspect
78 is very important: it ensures that if the PDU definitions are
79 extended after you finish your implementation (to accommodate
80 new versions of the protocol, say), you won't get into trouble with
81 uninitialized pointers in your structures. The functions use
82 <function>odr_malloc()</function> to
83 allocate the PDUs and its members, so you can free everything again with a
84 single call to <function>odr_reset()</function>. We strongly recommend
85 that you use the <literal>zget_*</literal>
86 functions whenever you are preparing a PDU (in a C++ API, the
87 <literal>zget_</literal>
88 functions would probably be promoted to constructors for the
89 individual types).
90 </para>
91 <para>
92 The prototype for the individual PDU types generally look like this:
93 </para>
94 <synopsis>
95   Z_&lt;type> *zget_&lt;type>(ODR o);
96 </synopsis>
97
98 <para>
99 eg.:
100 </para>
101
102 <synopsis>
103   Z_InitRequest *zget_InitRequest(ODR o);
104 </synopsis>
105
106 <para>
107 The &odr; handle should generally be your encoding stream, but it needn't be.
108 </para>
109 <para>
110 As well as the individual PDU functions, a function <function>
111 zget_APDU()</function> is
112 provided, which allocates a toplevel Z-APDU of the type requested:
113 </para>
114
115 <synopsis>
116   Z_APDU *zget_APDU(ODR o, int which);
117 </synopsis>
118
119 <para>
120 The <varname>which</varname> parameter is (of course) the discriminator
121 belonging to the <varname>Z_APDU</varname> <literal>CHOICE</literal> type.
122 All of the interface described here is provided by the &asn; module, and
123 you access it through the <filename>proto.h</filename> header file.
124
125 </para>
126 </sect1>
127 <sect1><title id="oid">Object Identifiers</title>
128 <para>
129 When you refer to object identifiers in your application, you need to
130 be aware that SR and Z39.50 use two different set of OIDs to refer to
131 the same objects. To handle this easily, &yaz; provides a utility module
132 to &asn; which provides an internal representation of the OIDs used in
133 both protocols. Each oid is described by a structure:
134 </para>
135
136 <screen>
137 typedef struct oident
138 {
139     enum oid_proto proto;
140     enum oid_class class;
141     enum oid_value value;
142     int oidsuffix[OID_SIZE];
143     char *desc;
144 } oident;
145 </screen>
146
147 <para>
148 The <literal>proto</literal> field can be set to either
149 <literal>PROTO_SR</literal> or <literal>PROTO_Z3950</literal>.
150 The <literal>class</literal> might be, say,
151 <literal>CLASS_RECSYN</literal>, and the <literal>value</literal> might be
152 <literal>VAL_USMARC</literal> for the USMARC record format. Functions
153 </para>
154
155 <screen>
156 int *oid_ent_to_oid(struct oident *ent, int *dst);
157 struct oident *oid_getentbyoid(int *o);
158 </screen>
159
160 <para>
161 are provided to map between object identifiers and database entries.
162 If you store a member of the <literal>oid_proto</literal> type in
163 your association state information, it's a simple matter, at runtime,
164 to generate the correct OID when you need it. For decoding, you can
165 simply ignore the proto field, or if you're strict, you can verify
166 that your peer is using the OID family from the correct protocol.
167 The <literal>desc</literal> field is a short, human-readable name
168 for the PDU, useful mainly for diagnostic output.
169 </para>
170
171 <note>
172 <para>
173 The old function <function>oid_getoidbyent</function> still exists but is
174 not thread safe. Use <function>oid_ent_to_oid</function> instead
175 and pass an array of size <literal>OID_SIZE</literal>.
176 </para>
177 </note>
178
179 <note>
180 <para>
181 Plans are underway to merge the two protocols into a single
182 definition, with one set of object identifiers. When this happens, the
183 oid module will no longer be required to support protocol
184 independence, but it should still be useful as a simple OID database.
185 </para>
186 </note>
187
188 </sect1>
189 <sect1><title>EXTERNAL Data</title>
190
191 <para>
192 In order to achieve extensibility and adaptability to different
193 application domains, the new version of the protocol defines many
194 structures outside of the main ASN.1 specification, referencing them
195 through ASN.1 EXTERNAL constructs. To simplify the construction and access
196 to the externally referenced data, the &asn; module defines a
197 specialized version of the EXTERNAL construct, called
198 <literal>Z_External</literal>.It is defined thus:
199 </para>
200
201 <screen>
202 typedef struct Z_External
203 {
204     Odr_oid *direct_reference;
205     int *indirect_reference;
206     char *descriptor;
207     enum
208     {
209         /* Generic types */
210         Z_External_single = 0,
211         Z_External_octet,
212         Z_External_arbitrary,
213
214         /* Specific types */
215         Z_External_SUTRS,
216         Z_External_explainRecord,
217         Z_External_resourceReport1,
218         Z_External_resourceReport2
219
220         ...
221
222     } which;
223     union
224     {
225         /* Generic types */
226         Odr_any *single_ASN1_type;
227         Odr_oct *octet_aligned;
228         Odr_bitmask *arbitrary;
229
230         /* Specific types */
231         Z_SUTRS *sutrs;
232         Z_ExplainRecord *explainRecord;
233         Z_ResourceReport1 *resourceReport1;
234         Z_ResourceReport2 *resourceReport2;
235
236         ...
237
238     } u;
239 } Z_External;
240 </screen>
241
242 <para>
243 When decoding, the &asn; module will attempt to determine which
244 syntax describes the data by looking at the reference fields
245 (currently only the direct-reference). For ASN.1 structured data, you
246 need only consult the <literal>which</literal> field to determine the type of
247 data. You can the access  the data directly through the union. When
248 constructing data for encoding, you set the union pointer to point to
249 the data, and set the <literal>which</literal> field accordingly.
250 Remember also to set the direct (or indirect) reference to the correct
251 OID for the data type.
252 For non-ASN.1 data such as MARC records, use the
253 <literal>octet_aligned</literal> arm of the union.
254 </para>
255
256 <para>
257 Some servers return ASN.1 structured data values (eg. database
258 records) as BER-encoded records placed in the <literal>octet-aligned</literal>
259 branch of the EXTERNAL CHOICE. The ASN-module will <emphasis>not</emphasis>
260 automatically decode these records. To help you decode the records in
261 the application, the function
262 </para>
263
264 <screen>
265 Z_ext_typeent *z_ext_gettypebyref(oid_value ref);
266 </screen>
267
268 <para>
269 Can be used to retrieve information about the known, external data
270 types. The function return a pointer to a static area, or NULL, if no
271 match for the given direct reference is found. The
272 <literal>Z_ext_typeent</literal>
273 is defined as:
274 </para>
275
276 <screen>
277 typedef struct Z_ext_typeent
278 {
279     oid_value dref;    /* the direct-reference OID value. */
280     int what;          /* discriminator value for the external CHOICE */
281     Odr_fun fun;       /* decoder function */
282 } Z_ext_typeent;
283 </screen>
284
285 <para>
286 The <literal>what</literal> member contains the <literal>Z_External</literal>
287 union discriminator value for the given type: For the SUTRS record
288 syntax, the value would be <literal>Z_External_sutrs</literal>.
289 The <literal>fun</literal> member contains a pointer to the
290 function which encodes/decodes the given type. Again, for the SUTRS
291 record syntax, the value of <literal>fun</literal> would be
292 <literal>z_SUTRS</literal> (a function pointer).
293 </para>
294
295 <para>
296 If you receive an EXTERNAL which contains an octet-string value that
297 you suspect of being an ASN.1-structured data value, you can use
298 <literal>z_ext_gettypebyref</literal> to look for the provided
299 direct-reference.
300 If the return value is different from NULL, you can use the provided
301 function to decode the BER string (see section <link linkend="odr-use">
302 Using ODR</link>).
303 </para>
304
305 <para>
306 If you want to <emphasis>send</emphasis> EXTERNALs containing
307 ASN.1-structured values in the occtet-aligned branch of the CHOICE, this
308 is possible too. However, on the encoding phase, it requires a somewhat
309 involved juggling around of the various buffers involved.
310 </para>
311 <para>
312 If you need to add new, externally defined data types, you must update
313 the struct above, in the source file <filename>prt-ext.h</filename>, as
314 well as the encoder/decoder in the file <filename>prt-ext.c</filename>.
315 When changing the latter, remember to update both the <literal>arm</literal>
316 arrary and the list <literal>type_table</literal>, which drives the CHOICE
317 biasing that is necessary to tell the different, structured types apart
318 on decoding.
319 </para>
320
321 <note>
322 <para>
323 Eventually, the EXTERNAL processing will most likely
324 automatically insert the correct OIDs or indirect-refs. First,
325 however, we need to determine how application-context management
326 (specifically the presentation-context-list) should fit into the
327 various modules.
328 </para>
329 </note>
330
331 </sect1>
332 <sect1><title>PDU Contents Table</title>
333
334 <para>
335 We include, for reference, a listing of the fields of each top-level
336 PDU, as well as their default settings.
337 </para>
338
339 <table frame="top"><title>Default settings for PDU Initialize Request</title>
340 <tgroup cols="3">
341 <colspec colname="field"></colspec>
342 <colspec colname="type"></colspec>
343 <colspec colname="value"></colspec>
344 <thead>
345 <row>
346 <entry>Field</entry>
347 <entry>Type</entry>
348 <entry>Default Value</entry>
349 </row>
350 </thead>
351 <tbody>
352
353 <row><entry>
354 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
355 </entry></row>
356
357 <row><entry>
358 protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
359 </entry></row>
360
361 <row><entry>
362 options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
363 </entry></row>
364
365 <row><entry>
366 preferredMessageSize</entry><entry>int</entry><entry>30*1024
367 </entry></row>
368
369 <row><entry>
370 maximumRecordSize</entry><entry>int</entry><entry>30*1024
371 </entry></row>
372
373 <row><entry>
374 idAuthentication</entry><entry>Z_IdAuthentication</entry><entry>NULL
375 </entry></row>
376
377 <row><entry>
378 implementationId</entry><entry>char*</entry><entry>"YAZ (id=81)"
379 </entry></row>
380
381 <row><entry>
382 implementationName</entry><entry>char*</entry><entry>"Index Data/YAZ"
383 </entry></row>
384
385 <row><entry>
386 implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
387 </entry></row>
388
389 <row><entry>
390 userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
391 </entry></row>
392
393 <row><entry>
394 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
395 </entry></row>
396
397 </tbody>
398 </tgroup>
399 </table>
400
401 <table frame="top"><title>Default settings for PDU Initialize Response</title>
402 <tgroup cols="3">
403 <colspec colname="field"></colspec>
404 <colspec colname="type"></colspec>
405 <colspec colname="value"></colspec>
406 <thead>
407 <row>
408 <entry>Field</entry>
409 <entry>Type</entry>
410 <entry>Default Value</entry>
411 </row>
412 </thead>
413 <tbody>
414
415 <row><entry>
416 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
417 </entry></row>
418
419 <row><entry>
420 protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
421 </entry></row>
422
423 <row><entry>
424 options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
425 </entry></row>
426
427 <row><entry>
428 preferredMessageSize</entry><entry>int</entry><entry>30*1024
429 </entry></row>
430
431 <row><entry>
432 maximumRecordSize</entry><entry>int</entry><entry>30*1024
433 </entry></row>
434
435 <row><entry>
436 result</entry><entry>bool_t</entry><entry>TRUE
437 </entry></row>
438
439 <row><entry>
440 implementationId</entry><entry>char*</entry><entry>"YAZ (id=81)"
441 </entry></row>
442
443 <row><entry>
444 implementationName</entry><entry>char*</entry><entry>"Index Data/YAZ"
445 </entry></row>
446
447 <row><entry>
448 implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
449 </entry></row>
450
451 <row><entry>
452 userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
453 </entry></row>
454
455 <row><entry>
456 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
457 </entry></row>
458
459 </tbody>
460 </tgroup>
461 </table>
462
463 <table frame="top"><title>Default settings for PDU Search Request</title>
464 <tgroup cols="3">
465 <colspec colname="field"></colspec>
466 <colspec colname="type"></colspec>
467 <colspec colname="value"></colspec>
468 <thead>
469 <row>
470 <entry>Field</entry>
471 <entry>Type</entry>
472 <entry>Default Value</entry>
473 </row>
474 </thead>
475 <tbody>
476
477 <row><entry>
478 referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
479 </entry></row>
480
481 <row><entry>
482 smallSetUpperBound</entry><entry>int</entry><entry>0
483 </entry></row>
484
485 <row><entry>
486 largeSetLowerBound</entry><entry>int</entry><entry>1
487 </entry></row>
488
489 <row><entry>
490 mediumSetPresentNumber</entry><entry>int</entry><entry>0
491 </entry></row>
492
493 <row><entry>
494 replaceIndicator</entry><entry>bool_t</entry><entry>TRUE
495 </entry></row>
496
497 <row><entry>
498 resultSetName</entry><entry>char *</entry><entry>"default"
499 </entry></row>
500
501 <row><entry>
502 num_databaseNames</entry><entry>int</entry><entry>0
503 </entry></row>
504
505 <row><entry>
506 databaseNames</entry><entry>char **</entry><entry>NULL
507 </entry></row>
508
509 <row><entry>
510 smallSetElementSetNames</entry><entry>Z_ElementSetNames</entry><entry>NULL
511 </entry></row>
512
513 <row><entry>
514 mediumSetElementSetNames</entry><entry>Z_ElementSetNames</entry><entry>NULL
515 </entry></row>
516
517 <row><entry>
518 preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
519 </entry></row>
520
521 <row><entry>
522 query</entry><entry>Z_Query</entry><entry>NULL
523 </entry></row>
524
525 <row><entry>
526 additionalSearchInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
527 </entry></row>
528
529 <row><entry>
530 otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
531 </entry></row>
532
533 </tbody>
534 </tgroup>
535 </table>
536
537 <screen>
538 Z_SearchResponse
539 ----------------
540 Field                        Type                Default value
541
542 referenceId                  Z_ReferenceId       NULL
543 resultCount                  int                 0
544 numberOfRecordsReturned      int                 0
545 nextResultSetPosition        int                 0
546 searchStatus                 bool_t              TRUE
547 resultSetStatus              int                 NULL
548 presentStatus                int                 NULL
549 records                      Z_Records           NULL
550 additionalSearchInfo         Z_OtherInformation  NULL
551 otherInfo                    Z_OtherInformation  NULL
552 </screen>
553
554 <screen>
555 Z_PresentRequest
556 ----------------
557 Field                        Type                Default value
558
559 referenceId                  Z_ReferenceId       NULL
560 resultSetId                  char*               "default"
561 resultSetStartPoint          int                 1
562 numberOfRecordsRequested     int                 10
563 num_ranges                   int                 0
564 additionalRanges             Z_Range             NULL
565 recordComposition            Z_RecordComposition NULL
566 preferredRecordSyntax        Odr_oid             NULL
567 maxSegmentCount              int                 NULL
568 maxRecordSize                int                 NULL
569 maxSegmentSize               int                 NULL
570 otherInfo                    Z_OtherInformation  NULL
571 </screen>
572
573 <screen>
574 Z_PresentResponse
575 -----------------
576 Field                        Type                Default value
577
578 referenceId                  Z_ReferenceId       NULL
579 numberOfRecordsReturned      int                 0
580 nextResultSetPosition        int                 0
581 presentStatus                int                 Z_PRES_SUCCESS
582 records                      Z_Records           NULL
583 otherInfo                    Z_OtherInformation  NULL
584 </screen>
585
586 <screen>
587 Z_DeleteResultSetRequest
588 ------------------------
589 Field                        Type                Default value
590
591 referenceId                  Z_ReferenceId       NULL
592 deleteFunction               int                 Z_DeleteRequest_list
593 num_ids                      int                 0
594 resultSetList                char**              NULL
595 otherInfo                    Z_OtherInformation  NULL
596 </screen>
597
598 <screen>
599 Z_DeleteResultSetResponse
600 -------------------------
601 Field                        Type                Default value
602
603 referenceId                  Z_ReferenceId       NULL
604 deleteOperationStatus        int                 Z_DeleteStatus_success
605 num_statuses                 int                 0
606 deleteListStatuses           Z_ListStatus**      NULL
607 numberNotDeleted             int                 NULL
608 num_bulkStatuses             int                 0
609 bulkStatuses                 Z_ListStatus        NULL
610 deleteMessage                char*               NULL
611 otherInfo                    Z_OtherInformation  NULL
612 </screen>
613
614 <screen>
615 Z_ScanRequest
616 -------------
617 Field                        Type                Default value
618
619 referenceId                  Z_ReferenceId       NULL
620 num_databaseNames            int                 0
621 databaseNames                char**              NULL
622 attributeSet                 Odr_oid             NULL
623 termListAndStartPoint        Z_AttributesPlus... NULL
624 stepSize                     int                 NULL
625 numberOfTermsRequested       int                 20
626 preferredPositionInResponse  int                 NULL
627 otherInfo                    Z_OtherInformation  NULL
628 </screen>
629
630 <screen>
631 Z_ScanResponse
632 --------------
633 Field                        Type                Default value
634
635 referenceId                  Z_ReferenceId       NULL
636 stepSize                     int                 NULL
637 scanStatus                   int                 Z_Scan_success
638 numberOfEntriesReturned      int                 0
639 positionOfTerm               int                 NULL
640 entries                      Z_ListEntris        NULL
641 attributeSet                 Odr_oid             NULL
642 otherInfo                    Z_OtherInformation  NULL
643 </screen>
644
645 <screen>
646 Z_TriggerResourceControlRequest
647 -------------------------------
648 Field                        Type                Default value
649
650 referenceId                  Z_ReferenceId       NULL
651 requestedAction              int                 Z_TriggerResourceCtrl_resou..
652 prefResourceReportFormat     Odr_oid             NULL
653 resultSetWanted              bool_t              NULL
654 otherInfo                    Z_OtherInformation  NULL
655 </screen>
656
657 <screen>
658 Z_ResourceControlRequest
659 ------------------------
660 Field                        Type                Default value
661
662 referenceId                  Z_ReferenceId       NULL
663 suspendedFlag                bool_t              NULL
664 resourceReport               Z_External          NULL
665 partialResultsAvailable      int                 NULL
666 responseRequired             bool_t              FALSE
667 triggeredRequestFlag         bool_t              NULL
668 otherInfo                    Z_OtherInformation  NULL
669 </screen>
670
671 <screen>
672 Z_ResourceControlResponse
673 -------------------------
674 Field                        Type                Default value
675
676 referenceId                  Z_ReferenceId       NULL
677 continueFlag                 bool_t              TRUE
678 resultSetWanted              bool_t              NULL
679 otherInfo                    Z_OtherInformation  NULL
680 </screen>
681
682 <screen>
683 Z_AccessControlRequest
684 ----------------------
685 Field                        Type                Default value
686
687 referenceId                  Z_ReferenceId       NULL
688 which                        enum                Z_AccessRequest_simpleForm;
689 u                            union               NULL
690 otherInfo                    Z_OtherInformation  NULL
691 </screen>
692
693 <screen>
694 Z_AccessControlResponse
695 -----------------------
696 Field                        Type                Default value
697
698 referenceId                  Z_ReferenceId       NULL
699 which                        enum                Z_AccessResponse_simpleForm
700 u                            union               NULL
701 diagnostic                   Z_DiagRec           NULL
702 otherInfo                    Z_OtherInformation  NULL
703 </screen>
704
705 <screen>
706 Z_Segment
707 ---------
708 Field                        Type                Default value
709
710 referenceId                  Z_ReferenceId       NULL
711 numberOfRecordsReturned      int                 value=0
712 num_segmentRecords           int                 0
713 segmentRecords               Z_NamePlusRecord    NULL
714 otherInfo                    Z_OtherInformation  NULL
715 </screen>
716
717 <screen>
718 Z_Close
719 -------
720 Field                        Type                Default value
721
722 referenceId                  Z_ReferenceId       NULL
723 closeReason                  int                 Z_Close_finished
724 diagnosticInformation        char*               NULL
725 resourceReportFormat         Odr_oid             NULL
726 resourceFormat               Z_External          NULL
727 otherInfo                    Z_OtherInformation  NULL
728
729 </screen>
730
731 </sect1>
732 </chapter>