Description of SUTRS.

[ir-tcl-moved-to-github.git] / doc / ir-tcl.sgml

<!doctype linuxdoc system>

<!--
  $Id: ir-tcl.sgml,v 1.9 1995-06-25 10:25:33 adam Exp $
-->

<article>
<title>IrTcl User's Guide and Reference
<author>Index Data, <tt/info@index.ping.dk/
<date>$Revision: 1.9 $
<abstract>
This document describes IrTcl &mdash; an information retrieval toolkit for
Tcl and Tk that provides access to the Z39.50/SR protocol.
</abstract>

<toc>

<sect>Introduction

<p>
This document describes the <sf/IrTcl/ information retrieval toolkit,
which offers a high-level, client interface to the Z39.50 and SR protocols.
The toolkit is based on the Tcl/Tk toolkit developed by Prof. John
K. Ousterhout at the University of California &lsqb;ref 1&rsqb;. 
Tcl is a simple, somewhat shell-like, interpreted language. What
makes Tcl attractive is that it also offers a C API, which makes
extensions to the language possible. The most important Tcl extension is
probably Tk &mdash; A Motif look-and-feel interface to the X window
system.

To interface the Z39.50/SR protocol <sf/IrTcl/ uses <bf/YAZ/.
<bf/YAZ/ offers two transport types: RFC1729/BER on TCP/IP and the mOSI
protocol stack.
However, the mOSI transport is only an option, and hence it is not
needed unless you wish to communicate within an OSI environment.
See &lsqb;ref 2&rsqb; for more information about the XTI/mOSI implementation.

<sf/IrTcl/ provides two system environments:

<itemize>
<item> A simple command line shell &mdash; useful for
testing purposes.
<item> A system which operates within the Tk environment which
makes it very easy to implement GUI clients.
</itemize>

<sect>Compilation and installation

<p>
In order to compile you need:
<itemize>
<item> An ANSI C compiler such as GNU C.
<item> Tcl 7.3. 
<item> YAZ version 1.0b or higher
</itemize>

As an option you may want:
<itemize>
<item> Tk 3.6.
<item> XTI/mosi
</itemize>

Newer versions of Tcl and Tk have been released. These packages
will <em/not/ work with <sf/IrTcl/. The <sf/IrTcl/ package will
probably be able to use the newer versions soon. Fortunately this
move will not change the <sf/IrTcl/ API &mdash; only the Tk code of the
test client will be modified.

Unpack the <sf/IrTcl/ package at the same directory level as <bf/YAZ/. 

Type:
<tscreen><verb>
$ ./configure
</verb></tscreen>

This command tries to configure <sf/IrTcl/ for your system and creates
a <tt>Makefile</tt>.

If the <tt>configure</tt> command cannot locate Tcl and Tk in your standard
locations for libraries searched by your C compiler it will guess 
that the libraries are located in <tt>/usr/local/lib</tt> and that 
the header files are located in <tt>/usr/local/include</tt>. 
If this is incorrect you will have to modify the <tt>Makefile</tt> yourself.

Compile <sf/IrTcl/ by typing:
<tscreen><verb>
$ make
</verb></tscreen>

If you don't have Tk you will only be able to create the <tt>ir-tcl</tt>
program and you must type <tt>make ir-tcl</tt> instead.

If successful, this will make <tt>ir-tcl</tt>, <tt>ir-tk</tt> (if
Tk is present) and a library called <tt>libirtcl.a</tt>.

To install the programs and support files type:
<tscreen><verb>
$ make install
</verb></tscreen>

Summary of files installed (the names refer to the Makefile variables):

<descrip>
<tag><tt>ir-tk</tt></tag> The Tk client. Installed in <tt>BINDIR</tt> &mdash;
defaults to <tt>/usr/local/bin</tt>. When ir-tk starts it reads
<tt>client.tcl</tt>. If the files doesn't exist in the current
directory it tries to read it from <tt>IRTCLDIR</tt> - defaults
to <tt>/usr/local/lib/irtcl</tt>.
<tag><tt>ir-tcl</tt></tag> The Tcl client. Installed in <tt>BINDIR</tt> &mdash;
defaults to <tt>/usr/local/bin</tt>.
<tag><tt>libirtcl.a</tt></tag> The <sf/IrTcl/ library. 
Installed in <tt>LIBDIR</tt> &mdash; defaults to <tt>/usr/local/lib</tt>.
<tag><tt>ir-tcl.h</tt></tag> The <sf/IrTcl/ header file.
Installed in <tt>INCDIR</tt> &mdash; defaults to <tt>/usr/local/include</tt>.
<tag><tt>client.tcl</tt></tag> A graphical client written in TK. 
Installed in <tt>IRTCLDIR</tt> &mdash; defaults to 
<tt>/usr/local/lib/irtcl</tt>.
<tag><tt>clientrc.tcl</tt></tag> A setup file with definitions
of target and queries. Read and updated by <tt>client.tcl</tt>. Installed
in <tt>IRTCLDIR</tt> &mdash; defaults to <tt>/usr/local/lib/irtcl</tt>.
<tag><tt>formats/*</tt></tag> Display format files written
in Tk. Read by <tt>client.tcl</tt>. Installed 
in <tt>IRTCLDIR</tt> &mdash; defaults to <tt>/usr/local/lib/irtcl</tt>.
<tag><tt>bitmaps/*</tt></tag> Various bitmap files. Read by
<tt>client.tcl</tt>. Installed 
in <tt>IRTCLDIR</tt> &mdash; defaults to <tt>/usr/local/lib/irtcl</tt>.
<tag><tt>LICENSE</tt></tag> LICENSE file. Read by
<tt>client.tcl</tt>. Installed 
in <tt>IRTCLDIR</tt> &mdash; defaults to <tt>/usr/local/lib/irtcl</tt>.
</descrip>

<sect1>ir-tcl

<p>
The <tt>ir-tcl</tt> program is a shell like <tt>tclsh</tt> except that
<tt>ir-tcl</tt> features the new set of information retrieval commands. 
Normally <tt>ir-tcl</tt> waits on <tt/stdin/ (for you to type commands) and
on sockets events (connected to Z39.50/SR targets).
You simply type the Tcl commands line by line. A filename may be specified as 
argument to <tt>ir-tcl</tt> in which case the file specified is evaluated 
as a script.

<sect1>ir-tk

<p>
<tt>ir-tk</tt> is a program which basically waits for X events and
socket events (assocated with Z39.50/SR targets). <tt>ir-tk</tt> normally
tries to evaluate the file named <tt>client.tcl</tt> when it
starts. However, this behaviour may be changed by specifying another 
filename with the <tt>-file</tt> option.

The enclosed script <tt>client.tcl</tt> is a graphical client which
demonstates an example of a user interface for the Z39.50/SR protocols.
At first the script was relatively small but it has grown since the 
beginning. At present it is about 3000 lines.

The client lets up define targets and query types within the interface.
Hence, you will not need to modify configation files. 

Stuff concerning targets can be found in the pull down menu 'Target'
with the following options:
<descrip>
<tag>Connect</tag> Establishes connection to a target.
<tag>Disconnect</tag> Closes a target connection.
<tag>About</tag> Shows implementation Id, implementation Version, etc 
 for the current target.
<tag>Setup</tag> Pops up a target definition window. You may alter
 a target definition.
<tag>Setup new</tag> Lets you define a new target.
</descrip> 

The term query type refers to a collection of search fields. The
pull down menu Options|Query deals with queries. You may
insert/modify/remove query types.

<sect>Overview of the API

<p>
Basically, <sf/IrTcl/ is a set of commands introduced to Tcl.
When extending Tcl there are two approaches: action-oriented commands
and object-oriented commands.

Action-oriented commands manipulate
Tcl variables and each command introduces only one action.
The string manipulation commands in Tcl are action oriented.

Object-oriented commands are added for every declared
variable (object). Object-oriented commands usually provide a set of
actions (methods) to manipulate the object.
The widgets in Tk (X objects) are examples of the object-oriented style.

<sf/IrTcl/ commands are object-oriented. The main reason
for this is that the data structures involved in the IR protocol
are not easily represented by Tcl data structures.
Also, the <sf/IrTcl/ objects tend to exist for a relativly long time.
Note that although we use the term object-oriented commands, this
does not mean that the programming style is strictly object-oriented. For
example, there is such no such thing as inheritance.

We are now ready to present the three commands introduced to Tcl by
<sf/IrTcl/:

<descrip>
<tag/ir/ The ir object represents a connection to a target. More
precisely it describes a Z-association. 
<tag/ir-set/ The ir-set describes a result set, which is
conceptually a collection of records returned by the target.
The ir-set object may retrieve records from a target by means of
the ir object; it may read/write records from/to a local file or it may be
updated with a user-edited record. 
<tag/ir-scan/ The scan object represents a list of scan lines
retrieved from a target.
</descrip>

<bf/Example/

To create a new IR object called <tt/z-assoc/ write:
<tscreen><verb>
   ir z-assoc
</verb></tscreen>

<bf/End of example/

Each object provides a set of <em/settings/ which may either be
readable, writeable of both. All settings immediately follow
the name of the object. If a value is present the setting
is set to <em/value/.

<bf/Example/

We wish to set the preferred-message-size to 18000 on the
<tt/z-assoc/ object:

<tscreen><verb>
   z-assoc preferredMessageSize 18000
</verb></tscreen>

To read the current value of preferred-message-size use:

<tscreen><verb>
   z-assoc preferredMessageSize
</verb></tscreen>
<bf/End of example/

One important category consists of settings is those that relate to the
event-driven model. When <sf/IrTcl/ receives responses from the target, i.e.
init responses, search responses, etc., a <em/callback/ routine
is called. Callback routines are represented in Tcl as
a list, which is re-interpreted prior to invocation.
The method is similar to the one used in Tk to capture X events.

For each SR/Z39.50 request there is a corresponding object action. The most
important actions are:
<descrip>
<tag/connect/ Establishes connection with a target
<tag/init/ Sends an initialize request.
<tag/search/ Sends a search request.
<tag/present/ Sends a present request.
<tag/scan/ Sends a scan request.
</descrip>

<bf/Example/

This example shows a complete connect - init - search - present scenario. 

First an IR object, called <tt/z/, is created.
Also a result set <tt/z.1/ is introduced by the <tt/ir-set/ 
and it is specified that the result set uses <tt/z/ as its association.

The setting <tt/databaseNames/ is set to the 
database <tt/books/ to which the following searches are directed.
A callback is then defined and a connection is established to
<tt/fake.com/ by the <tt/connect/ action.
If the connect succeeds the <tt/connect-response/ is called.

In the Tcl procedure, <tt/connect-response/, a callback is defined
<em/before/ the init request is executed. 
The Tcl procedure <tt/init-response/ is called when a 
init response is returned from the target.

The <tt/init-response/ procedure sets up a <tt/search-response/ 
callback handler and sends a search-request by using a query which
consists of a single word <tt/science/.

When the <tt/search-response/ procedure is called it defines
a variable <tt/hits/ and sets it to the value of the setting
<tt/resultCount/. If <tt/hits/ is positive a present-request is
sent &mdash; asking for 5 records from position 1. 

Finally, a present response is received and the number of records
returned is stored in the variable <tt/ret/.

<tscreen><verb>
ir z
ir-set z.1 z
z databaseNames books
z callback {connect-response}
z connect fake.com

proc connect-response {} {
    z callback {init-response}
    z init
}

proc init-response {} {
    z.1 callback {search-response}
    z.1 search science
}

proc search-response {} {
    set hits [z.1 resultCount]
    puts "$hits hits"
    if {$hits > 0} {
        z.1 callback {present-response}
        z.1 present 1 5
    }
}

proc present-response {} {
    set ret [z.1 numberOfRecordsReturned]
    puts "$ret records returned"
}
</verb></tscreen>
<bf/End of example/

The previous example program doesn't care about error conditions. 
If errors occur in the program they will be trapped by the Tcl error 
handler. This is not always appropriate. However, Tcl offers a 
<tt/catch/ command to support error handling by the program itself.

<sect>Associations

<p>
The ir object describes an association with a target.
This section covers the connect-init-disconnect actions provided
by the ir object.
An ir object is created by the <tt/ir/ command and the
created object enters a 'not connected' state, because it isn't
connected to a target yet. 

<sect1>Connect

<p>
A connection is established by the <tt/connect/ action which is
immediately followed by a hostname. A number of settings affect the
<tt/connect/ action. Obviously, these settings should be set
<bf/before/ connecting. The settings are:

<descrip>
<tag><tt>comstack </tt><tt>mosi|tcpip</tt></tag>
 Comstack type.
<tag><tt>protocol </tt><tt>Z39|SR</tt></tag>
 Protocol type - ANSI/NISO Z39.50 or ISO SR.
<tag><tt>callback </tt><em>list</em></tag>
 Tcl script called when the connection is established
<tag><tt>failback </tt><em>list</em></tag>
 Fatal error Tcl script. Called on protocol errors or if target
 closes connection
</descrip>

If the connect is unsuccessful either the connect action itself
will return an error code or the failback handler is invoked.

<sect1>Init

<p>
If the connect operation succeeds the <tt/init/ action should be used. 
The init related settings are:

<descrip>
<tag><tt>preferredMessageSize </tt><em>integer</em></tag>
 Preferred-message-size. Default value is 30000.
<tag><tt>maximumRecordSize </tt><em>integer</em></tag>
 Maximum-record-size. Default value is 30000.
<tag><tt>idAuthentication </tt><em>string</em> ...</tag>
 Id-authentication. There are three forms. If any empty is
 given, the Id-authentication is not used.  If one non-empty string
 is given, the 'open' authentication is used. If three strings are 
 specified, the version 'id-pass' authentication (version 3 only) 
 is used in which case the first string is groupId; the second string 
 is userId and the third string is password.
<tag><tt>implementationName </tt><em>string</em></tag>
 Implementation-name of origin system. 
<tag><tt>implementationId</tt></tag>
 Implementation-id of origin system. This setting is read-only.
<tag><tt>implementationVersion</tt></tag>
 Implementation-version of origin system. This settings is read-only.
<tag><tt>options </tt><em>list</em></tag>
 Options to be negotiated in the init service. The list contains 
 the options that are set. Possible values are <tt>search</tt>, 
 <tt>present</tt>, <tt>delSet</tt>, <tt>resourceReport</tt>, 
 <tt>triggerResourceCtrl</tt>, <tt>resourceCtrl</tt>,
 <tt>accessCtrl</tt>, <tt>scan</tt>, <tt>sort</tt>, 
 <tt>extendedServices</tt>, <tt>level-1Segmentation</tt>, 
 <tt>level-2Segmentation</tt>, <tt>concurrentOperations</tt> and
 <tt>namedResultSets</tt>. Currently the default options are:
 <tt>search</tt>, <tt>present</tt>, <tt>scan</tt> and 
 <tt>namedResultSets</tt>. The <tt>options</tt> setting is set to its default
 value when an ir object is created and when a <tt>disconnect</tt>
 action is performed.
<tag><tt>protocolVersion </tt><em>integer</em></tag>
 Protocol version: 2, 3, etc. Default is 2.
<tag><tt>initResponse </tt><em>list</em></tag>
 Init-response Tcl script. Note: not implemented - use <tt>callback</tt>
 instead.
<tag><tt>callback </tt><em>list</em></tag>
 General response Tcl script. Only used if <tt>initResponse</tt>
 is not specified.
</descrip>

The init-response handler should inspect some of the settings shown
below:

<descrip>
<tag><tt>initResult </tt><em>boolean</em></tag>
 Init response status. True if init operation was successful; 
 false otherwise.
<tag><tt>preferredMessageSize </tt><em>integer</em></tag>
 Preferred-message-size.
<tag><tt>maximumRecordSize </tt><em>integer</em></tag>
 Maximum-record-size.
<tag><tt>targetImplementationName </tt><em>string</em></tag>
 Implementation-name of target system.
<tag><tt>targetImplementationId </tt><em>string</em></tag>
 Implementation-id of target system.
<tag><tt>targetImplementationVersion </tt><em>string</em></tag>
 Implementation-version of target system.
<tag><tt>options </tt><em>list</em></tag>
 Options negotiated in init. The list contains the options that are set.
<tag><tt>protocolVersion </tt><em>integer</em></tag>
 Protocol version: 2, 3, etc.
<tag><tt>userInformationField </tt><em>string</em></tag>
 User information field.
</descrip>

<bf/Example/

Consider a client with the ability to access multiple targets.

We define a list of targets that we wish to connect to.
Each item in the list describes the target parameters with
the following four components: association-name, comstack-type,
protocol-type and a hostname.

The list for the two targets: ISO/SR target DANBIB and TCP/Z39.50 
target Data Research, will be defined as:
<tscreen><verb>
set targetList { {danbib mosi SR 0103/find2.denet.dk:4500}
                 {drs tcpip Z39 dranet.dra.com} }
</verb></tscreen>

The Tcl code below defines, connect and initialize the
targets in <tt/targetList/:

<tscreen><verb>
foreach target $targetList {
    set assoc [lindex $target 0]
    ir $assoc
    $assoc comstack [lindex $target 1]
    $assoc protocol [lindex $target 2]
    $assoc failback [list fail-response $assoc]
    $assoc callback [list connect-response $assoc]
    $assoc connect [lindex $target 3]
}

proc connect-response {assoc} {
    $assoc callback [list init-response $assoc]
    $assoc init
}

proc fail-response {assoc} {
    puts "$assoc closed connection or protocol error"
}

proc init-response {assoc} {
    if {[$assoc initResult]} {
        puts "$assoc initialized ok"
    } else {
        puts "$assoc didn't initialize"
    }
}
</verb></tscreen>

<tt/target/ is bound to each item in the list of targets.
The <tt/assoc/ is set to the ir object name.
Then, the comstack, protocol and failback are set for the <tt/assoc/ object.
The ir object name is argument to the <tt/fail-response/ and 
<tt/connect-response/ routines. 
Note the use of the Tcl <tt/list/ command which 
is necessary here because the argument contains variables 
(<tt/assoc/) that should be substituted before the handler is defined.
After the connect operation, the <tt/init-response/ handler
is defined in much the same way as the failback handler.
And, finally, an init request is executed.

<bf/End of example/

<sect1>Disconnect

<p>
To terminate the connection the <tt/disconnect/ action should be used. 
This action has no parameters. 
Another connection may be established by a new <tt/connect/ action on
the same ir object.

<sect>Result sets

<p>
This section covers the queries used by <sf/IrTcl/, and how searches and
presents are handled.

A search operation and a result set is described by the ir set object.
The ir set object is defined by the <tt/ir-set/ command which
has two parameters. The first is the name of the new ir set object, and
the second, which is optional, is the name of an assocation &mdash; an ir
object. The second argument is required if the ir set object should be able
to perform searches and presents. However, it is not required if
only ``local'' operations is done with the ir set object.

When the ir set object is created a number of settings are inherited
from the ir object, such as the selected databass, query type,
etc. Thus, the ir object contains what we could call default
settings.

<sect1>Queries

<p>
Search requests are sent by the <tt/search/ action which
takes a query as parameter. There are two types of queries,
RPN and CCL, controlled by the setting <tt/queryType/.
A string representation for the query is used in <sf/IrTcl/ since
Tcl has reasonably powerful string manipulaton capabilities.
The RPN query used in <sf/IrTcl/ is the prefix query notation also used in
the <bf/YAZ/ test client.

The CCL query is an uninterpreted octet-string which is parsed by the target.
We refer to the standard: ISO 8777. Note that only a few targets
actually support the CCL query and the interpretation of
the standard may vary.

The prefix query notation (which is converted to RPN) offer a few
operators. They are:

<descrip>
<tag><tt>@attr </tt><em>list op</em></tag>
 The attributes in list are applied to op
<tag><tt>@and </tt><em>op1 op2</em></tag>
 Boolean <em/and/ on op1 and op2
<tag><tt>@or </tt><em>op1 op2</em></tag>
 Boolean <em/or/ on op1 and op2
<tag><tt>@not </tt><em>op1 op2</em></tag>
 Boolean <em/not/ on op1 and op2
<tag><tt>@prox </tt><em>list op1 op2</em></tag>
 Proximity operation on op1 and op2. Not implemented yet.
<tag><tt>@set </tt><em>name</em></tag>
 Result set reference
</descrip>

It is simple to build RPN queries in <sf/IrTcl/. Search terms
are sequences of characters, as in:
<tscreen><verb>
   science
</verb></tscreen>

Boolean operators use the prefix notation (instead of the suffix/RPN),
as in:
<tscreen><verb>
   @and science technology
</verb></tscreen>

Search terms may be associated with attributes. These
attributes are indicated by the <tt/@attr/ operator.
Assuming the bib-1 attribute set, we can set the use-attribute 
(type is 1) to title (value is 4):

<tscreen><verb>
   @attr 1=4 science
</verb></tscreen>

Also, it is possible to apply attributes to a range of search terms.
In the query below, both search terms have use=title but the <tt/tech/
term is right truncated:

<tscreen><verb>
   @attr 1=4 @and @attr 5=1 tech beta
</verb></tscreen>

<sect1>Search

<p>
The settings that affect the search are listed below:

<descrip>
<tag><tt>databaseNames </tt><em>list</em></tag>
 database-names.
<tag><tt>smallSetUpperBound </tt><em>integer</em></tag>
 small set upper bound. Default 0.
<tag><tt>largeSetLowerBound </tt><em>integer</em></tag>
 large set lower bound. Default 2.
<tag><tt>mediumSetPresentNumber </tt><em>integer</em></tag>
 medium set present number. Default 0.
<tag><tt>replaceIndicator </tt><em>boolean</em></tag>
 replace-indicator.
<tag><tt>setName </tt><em>string</em></tag>
 name of result set.
<tag><tt>queryType rpn|ccl</tt></tag>
 query type-1 or query type-2 
<tag><tt>preferredRecordSyntax </tt><em>string</em></tag>
 preferred record syntax &mdash; UNIMARC, USMARC, etc. 
<tag><tt>smallSetElementSetNames </tt><em>string</em></tag>
 small-set-element-set names. Not implemented yet.
<tag><tt>mediumSetElementSetNames </tt><em>string</em></tag>
 medium-set-element-set names. Not implemented yet.
<tag><tt>searchResponse </tt><em>list</em></tag>
 Search-response Tcl script. Not implemented yet. Use <tt>callback</tt>
 instead.
<tag><tt>callback </tt><em>list</em></tag>
 General response Tcl script. Only used if searchResponse is not specified 
</descrip>

Setting the <tt/databaseNames/ is mandatory. All other settings
have reasonable defaults.
The search-response handler, specified by the <tt/callback/ - or
the <tt/searchResponse/ setting, 
should read some of the settings shown below:

<descrip>
<tag><tt>searchStatus </tt><em>boolean</em></tag>
 search-status. True if search operation was successful; false
 otherwise.
<tag><tt>responseStatus </tt><em>list</em></tag>
 response status information.
<tag><tt>resultCount </tt><em>integer</em></tag>
 result-count
<tag><tt>numberOfRecordsReturned </tt><em>integer</em></tag>
 number of records returned.
</descrip>

The <tt/responseStatus/ signals one of three conditions which
is indicated by the value of the first item in the list:

<descrip>
<tag><tt>NSD</tt></tag> indicates that the target has returned one or 
more non-surrogate diagnostic messages. The <tt/NSD/ item is followed by 
a list with all non-surrogate messages. Each non-surrogate message consists
of three items. The first item of the three items is the error
code (integer); the next item is a textual representation of the error
code in plain english; the third item is additional information, possibly
empty if no additional information was returned by the target.

<tag><tt>DBOSD</tt></tag> indicates a successful operation where the
target has returned one or more records. Each record may be
either a database record or a surrogate diagnostic.

<tag><tt>OK</tt></tag> indicates a successful operation &mdash; no records are
returned from the target. 
</descrip>

<bf/Example/

We continue with the multiple-targets example. 
The <tt/init-response/ procedure will attempt to make searches:

<tscreen><verb>
proc init-response {assoc} {
    puts "$assoc connected"
    ir-set ${assoc}.1 $assoc
    $assoc.1 queryType rpn
    $assoc.1 databaseNames base-a base-b
    $assoc.1 callback [list search-response $assoc ${assoc}.1]
    $assoc.1 search "@attr 1=4 @and @attr 5=1 tech beta"
}
</verb></tscreen>

An ir set object is defined and the
ir object is told about the name of ir object.
The ir set object use the name of the ir object as prefix.

Then, the query-type is defined to be RPN, i.e. we will
use the prefix query notation later on.

Two databases, <tt/base-a/ and <tt/base-b/, are selected.

A <tt/search-response/ handler is defined with the
ir object and the ir-set object as parameters and
the search is executed.

The first part of the <tt/search-response/ looks like:
<tscreen><verb>
proc search-response {assoc rset} {
    set status [$rset responseStatus]
    set type [lindex $status 0]
    if {$type == "NSD"} {
        set code [lindex $status 1]
        set msg [lindex $status 2]
        set addinfo [lindex $status 3]
        puts "NSD $code: $msg: $addinfo"
        return
    } 
    set hits [$rset resultCount]
    if {$type == "DBOSD"} {
        set ret [$rset numberOfRecordsReturned]
        ...
    }
}
</verb></tscreen>
The response status is stored in variable <tt/status/ and 
the first element indicates the condition.
If non-surrogate diagnostics are returned they are displayed.
Otherwise, the search was a success and the number of hits
is read. Finally, it is tested whether the search response
returned records (database or diagnostic).

Note that we actually didn't inspect the search status (setting
<tt/searchStatus/) to determine whether the search was successful or not, 
because the standard specifies that one or more non-surrogate
diagnostics should be returned by the target in case of errors.

<bf/End of example/

If one or more records are returned from the target they
will be stored in the result set object.
In the case in which the search response contains records, it is
very similar to the present response case. Therefore, some settings
are common to both situations.

<sect1>Present

<p>
The <tt/present/ action sends a present request. The <tt/present/ is
followed by two optional integers. The first integer is the 
result-set starting position &mdash; defaults to 1. The second integer 
is the number of records requested &mdash; defaults to 10. 
The settings which could be modified before a <tt/present/
action are:

<descrip>
<tag><tt>preferredRecordSyntax </tt><em>string</em></tag>
 preferred record syntax &mdash; UNIMARC, USMARC, etc.
<tag><tt>elementSetElementSetNames </tt><em>string</em></tag>
 element-set names 
<tag><tt>presentResponse </tt><em>list</em></tag>
 Present-response Tcl script. Not implemented yet. Use <tt>callback</tt>
 instead.
<tag><tt>callback </tt><em>list</em></tag>
 General response Tcl script. Only used if presentResponse is not specified 
</descrip>

The present-response handler should inspect the settings
shown in table below.
Note that <tt/responseStatus/ and <tt/numberOfRecordsReturned/
settings were also used in the search-response case.

As in the search response case, records returned from the
target are stored in the result set object.

<descrip>
<tag><tt>presentStatus </tt><em>boolean</em></tag>
 present-status 
<tag><tt>responseStatus </tt><em>list</em></tag>
 Response status information 
<tag><tt>numberOfRecordsReturned </tt><em>integer</em></tag>
 number of records returned 
<tag><tt>nextResultSetPosition </tt><em>integer</em></tag>
 next result set position
</descrip>

<sect1>Records

<p>
Search responses and present responses may result in
one or more records stored in the ir set object if
the <tt/responseStatus/ setting indicates database or
surrogate diagnostics (<tt/DBOSD/). The individual
records, indexed by an integer position, should be 
inspected.

The action <tt/type/ followed by an integer returns information 
about a given position in an ir set. There are three possiblities:

<descrip>
<tag><tt/SD/</tag> The item is a surrogate diagnostic record.
<tag><tt/DB/</tag> The item is a database record.
<tag><em/empty/</tag> There is no record at the specified position.
</descrip>

To handle the first case, surrogate diagnostic record, the
<tt/Diag/ action should be used. It returns three
items: error code (integer), text representation in plain english
(string), and additional information (string, possibly empty).

In the second case, database record, the <tt/recordType/ action should
be used. It returns the record type at the given position.
Some record types are:

<tscreen>
UNIMARC
INTERMARC
CCF
USMARC
UKMARC
NORMARC
LIBRISMARC
DANMARC
FINMARC
SUTRS
</tscreen>

<bf/Example/

We continue our search-response example. In the case,
<tt/DBOSD/, we should inspect the result set items. 
Recall that the ir set name was passed to the
search-response handler as argument <tt/rset/.

<tscreen><verb>
    if {$type == "DBOSD"} {
        set ret [$rset numberOfRecordsReturned]
        for {set i 1} {$i<=$ret} {incr i} {
            set itype [$rset type $i]
            if {$itype == "SD"} {
                set diag [$rset Diag $i]
                set code [lindex $diag 0]
                set msg [lindex $diag 1]
                set addinfo [lindex $diag 2]
                puts "$i: NSD $code: $msg: $addinfo"
            } elseif {$itype == "DB"} {
                set rtype [$rset recordType $i]
                puts "$i: type is $rtype"
            }
        }
    }
</verb></tscreen>
Each item in the result set is examined. 
If an item is a diagnostic message it is displayed; otherwise
if it's a database record its type is displayed.

<bf/End of example/

<sect1>MARC records

<p>
In the case, where there is a MARC record at a given position we
want to display it somehow. The action <tt/getMarc/ is what we need.
The <tt/getMarc/ is followed by a position integer and the type of
extraction we want to make: <tt/field/ or <tt/line/.

The <tt/field/ and <tt/line/ type are followed by three
parameters that serve as extraction masks.
They are called tag, indicator and field.
If the mask matches a tag/indicator/field of a record the information
is extracted. Two characters have special meaning in masks: the
dot (any character) and star (any number of any character).

The <tt/field/ type returns one or more lists of field information
that matches the mask specification. Only the content of fields
is returned.

The <tt/line/ type, on the other hand, returns a Tcl list that
completely describe the layout of the MARC record &mdash; including
tags, fields, etc.

The <tt/field/ type is sufficient and efficient in the case, where only a
small number of fields are extracted, and in the case where no
further processing (in Tcl) is necessary.

However, if the MARC record is to be edited or altered in any way, the
<tt/line/ extraction is more powerful &mdash; only limited by the Tcl
language itself.

<bf/Example/

Consider the record below:
<tscreen><verb>
001       11224466 
003    DLC
005    00000000000000.0
008    910710c19910701nju           00010 eng  
010    $a    11224466 
040    $a DLC $c DLC
050 00 $a 123-xyz
100 10 $a Jack Collins
245 10 $a How to program a computer
260 1  $a Penguin
263    $a 8710
300    $a p. cm.
</verb></tscreen>

Assuming this record is at position 1 in ir-set <tt/z.1/, we
might extract the title-field (245 * a), with the following command:
<tscreen><verb>
z.1 getMarc 1 field 245 * a
</verb></tscreen>

which gives:
<tscreen><verb>
{How to program a computer}
</verb></tscreen>

Using the <tt/line/ instead of <tt/field/ gives:
<tscreen><verb>
{245 {10} {{a {How to program a computer}} }}
</verb></tscreen>

If we wish to extract the whole record as a list, we use:
<tscreen><verb>
z.1 getMarc 1 line * * *
</verb></tscreen>

giving:
<tscreen><verb>
{001 {} {{{} {   11224466 }} }}
{003 {} {{{} DLC} }}
{005 {} {{{} 00000000000000.0} }}
{008 {} {{{} {910710c19910701nju           00010 eng  }} }}
{010 {  } {{a {   11224466 }} }}
{040 {  } {{a DLC} {c DLC} }}
{050 {00} {{a 123-xyz} }}
{100 {10} {{a {Jack Collins}} }}
{245 {10} {{a {How to program a computer}} }}
{260 {1 } {{a Penguin} }}
{263 {  } {{a 8710} }}
{300 {  } {{a {p. cm.}} }}
</verb></tscreen>

<bf/End of example/

<bf/Example/

This example demonstrates how Tcl can be used to examine
a MARC record in the list notation.

The procedure <tt/extract-format/ makes an extraction of
fields in a MARC record based on a number of masks.
There are 5 parameters, <tt/r/: a
record in list notation, <tt/tag/: regular expression to
match the record tags, <tt/ind/: regular expression to
match indicators, <tt/field/: regular expression to 
match fields, and finally <tt/text/: regular expression to 
match the content of a field.

<tscreen><verb>
proc extract-format {r tag ind field text} {
    foreach line $r {
        if {[regexp $tag [lindex $line 0]] && \
                [regexp $ind [lindex $line 1]]} {
            foreach f [lindex $line 2] {
                if {[regexp $field [lindex $f 0]]} {
                    if {[regexp $text [lindex $f 1]]} {
                        puts [lindex $f 1]
                    }
                }
            }
        }
    }
}
</verb></tscreen>

To match <tt/comput/ followed by any number of character(s) in the
245 fields in the record from the previous example, we could use:
<tscreen><verb>
set r [z.1 getMarc 1 line * * *]

extract-format $r 245 .. . comput
</verb></tscreen>
which gives:
<tscreen><verb>
How to program a computer
</verb></tscreen>

<bf/End of example/

The <tt/putMarc/ action does the opposite of <tt/getMarc/. It
copies a record in Tcl list notation to a ir set object and is
needed if a result-set must be updated by a Tcl modified (user-edited)
record.

<sect1>SUTRS

In <sf/IrTcl/ a SUTRS record is treated as one single string. To retrieve
a SUTRS string at a given index, the <tt>getSutrs</tt> should be used. 
The <tt>getSutrs</tt> is immediately followed by a index.

<sect>Scan

<p>
To perform scan, a scan object must be created by the <tt>ir-scan</tt>
command. This command has two arguments - name of the scan object and
name of the ir object. Basically, the scan object, provides one <tt>scan</tt>
action which sends a scan request to the target. The <tt>action</tt>
is followed by a string describing starting point of the term list. The
format used is a simple subset of the query used in search requests. Only
<tt>@attr</tt> specifications and simple terms are allowed.
The settings that affect the scan are:

<descrip>
<tag><tt>stepSize </tt><em>integer</em></tag>
 Step size. Default is 0.
<tag><tt>numberOfTermsRequested </tt><em>integer</em></tag>
 Number of terms requested. Default is 20.
<tag><tt>preferredPositionInResponse </tt><em>integer</em></tag>
 Preferred position in response. Default is 1.
<tag><tt>databaseNames </tt><em>list</em></tag>
 Database names. Note that this setting is not (yet) supported for
 the scan object. You must set this for the ir object instead.
<tag><tt>callback </tt><em>list</em></tag>
 General response Tcl script. This setting is not (yet) supported for
 the scan object. You must set this for the ir object instead.
</descrip>

The scan object normally holds one or more scan line entries upon
successful completion. The table below summarizes the settings
that should be used in a response handler.

<descrip>
<tag><tt>scanStatus</tt></tag>
 Scan status. An integer between 0 and 6.
<tag><tt>numberOfTermsReturned </tt><em>integer</em></tag>
 Number of terms returned. 
<tag><tt>positionOfTerm</tt></tag>
 An integer describing the position of term.
<tag><tt>scanLine </tt> <em>integer</em></tag>
 This function returns information about a given scan line (entry) at a given
 index specified by the integer. The first scan line is numbered zero;
 the second 1 and so on. A list is returned by the <tt>scanLine</tt>
 setting. The first element is <tt>T</tt> if the scan line
 is a normal term and <tt>SD</tt> if the scan line is a surrogate 
 diagnostic. In the first case (normal) the scan term is second element 
 in the list and the number of occurences is the third element.
 In the other case (surrogate diagnostic), the second element
 is the diagnostic code, the third a text representation of the error
 code and the fourth element is additional information.
</descrip>

<bf/Example/

We will scan for the terms after <tt>science</tt> in the Title index.
We will assume that an ir object called <tt>z-assoc</tt> has already
been created.

<tscreen><verb>
   z-assoc callback {scan-response}
   ir-scan z-scan z-assoc
   z-scan scan "@attr 1=4 science"
  
   proc scan-response {} {
       set status [z-scan status]
       if {$status == 0} {
           set no [z-scan numberOfTermsReturned]
           for {set i 0} {$i < $no} {incr i} {
               set line [z-scan scanLine $i]
               set type [lindex $line 0]
               if {$type == "T"} {
                   puts [lindex $line 1]
               } elseif {$type == "SD"} {
                   puts [lindex $line 1]
               }
           }
       }
   }
</verb></tscreen>
<bf/End of examle/

<sect>License

<p>
Copyright &copy; 1995, Index Data.

Permission to use, copy, modify, distribute, and sell this software and
its documentation, in whole or in part, for any purpose, is hereby granted,
provided that:

1. This copyright and permission notice appear in all copies of the
software and its documentation. Notices of copyright or attribution
which appear at the beginning of any file must remain unchanged.

2. The names of Index Data or the individual authors may not be used to
endorse or promote products derived from this software without specific
prior written permission.

THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTY OF ANY KIND,
EXPRESS, IMPLIED, OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY
WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
IN NO EVENT SHALL INDEX DATA BE LIABLE FOR ANY SPECIAL, INCIDENTAL,
INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR
NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF
LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
OF THIS SOFTWARE.

<sect>About Index Data

<p>
Index Data is a consulting and software-development enterprise that
specialises in library and information management systems. Our
interests and expertise span a broad range of related fields, and one
of our primary, long-term objectives is the development of a powerful
information management
system with open network interfaces and hypermedia capabilities.

We make this software available free of charge, on a fairly unrestrictive
license; as a service to the networking community, and to further the
development of quality software for open network communication.

We'll be happy to answer questions about the software, and about ourselves
in general.

<tscreen>
Index Data&nl
Ryesgade 3&nl
2200 K&oslash;benhavn N&nl
DENMARK
</tscreen>

<p>
<tscreen><verb>
Phone: +45 3536 3672
Fax  : +45 3536 0449
Email: info@index.ping.dk
</verb></tscreen>

<sect>References

<p>

<descrip>
<tag>1 Ousterhout, John K.:</tag>
Tcl and the Tk Toolkit. Addison-Wesley Company Inc (ISBN
0-201-63337-X). Source and documentation
can be found in <tt>URL:ftp://ftp.cs.berkeley.edu/pub/tcl</tt>
and mirrors.
<tag>2 Furniss, Peter:</tag>
RFC 1698: Octet Sequences for Upper-Layer OSI to Support
Basic Communications Applications.
</descrip>

</article>