X-Git-Url: http://git.indexdata.com/?p=yaz-moved-to-github.git;a=blobdiff_plain;f=doc%2Ffrontend.xml;h=985f4bd18348a9985b74ea7db56e3f8b4afcef5a;hp=bc6f9786b5e3424b9db2851bd8286d96eb1f0436;hb=95d8bd04e10519a635972a24176270ef4dbe8d2c;hpb=ce853cc4919ab346fd629e7727905d3ee6e1129f diff --git a/doc/frontend.xml b/doc/frontend.xml index bc6f978..985f4bd 100644 --- a/doc/frontend.xml +++ b/doc/frontend.xml @@ -1,22 +1,19 @@ - - Making an IR Server for Your Database - + + Generic server Introduction - + If you aren't into documentation, a good way to learn how the - backend interface works is to look at the backend.h + back end interface works is to look at the backend.h file. Then, look at the small dummy-server in - ztest/ztest.c. Finally, you can have a look at - the seshigh.c file, which is where most of the - logic of the frontend server is located. The backend.h + ztest/ztest.c. The backend.h file also makes a good reference, once you've chewed your way through the prose of this file. If you have a database system that you would like to make available by - means of Z39.50, &yaz; basically offers your two options. You + means of Z39.50, SRW o SRU, &yaz; basically offers your two options. You can use the APIs provided by the &asn;, &odr;, and &comstack; modules to create and decode PDUs, and exchange them with a client. @@ -35,18 +32,24 @@ The backend interface was designed in anticipation of a specific integration task, while still attempting to achieve some degree of - generality. We realise fully that there are points where the + generality. We realize fully that there are points where the interface can be improved significantly. If you have specific functions or parameters that you think could be useful, send us a mail (or better, sign on to the mailing list referred to in the - toplevel README file). We will try to fit good suggestions into future + top-level README file). We will try to fit good suggestions into future releases, to the extent that it can be done without requiring too many structural changes in existing applications. - - The Database Frontend + + + The &yaz; server does not support XCQL. + + + + + The Database Frontend We refer to this software as a generic database frontend. Your @@ -55,7 +58,8 @@ The backend API consists of a small number of function handlers and structure definitions. You are required to provide the main() routine for the server (which can be - quite simple), as well as a set of handlers to match each of the prototypes. + quite simple), as well as a set of handlers to match each of the + prototypes. The interface functions that you write can use any mechanism you like to communicate with your database system: You might link the whole thing together with your database application and access it by @@ -86,7 +90,7 @@ Extended Services (optional). - + Result-Set Delete (optional). @@ -94,7 +98,11 @@ Result-Set Sort (optional). - + + + Return Explain for SRW/SRU (optional). + + @@ -103,22 +111,22 @@ - The Backend API + The Backend API - The headers files that you need to use the interface are in the - include/yaz directory. They are called - statserv.h and backend.h. They - will include other files from the include/yaz - directory, so you'll probably want to use the -I option of your - compiler to tell it where to find the files. When you run - make in the toplevel &yaz; directory, - everything you need to create your server is put the - lib/libyaz.a library. + The header file that you need to use the interface are in the + include/yaz directory. It's called + backend.h. It will include other files from + the include/yaz directory, so you'll + probably want to use the -I option of your compiler to tell it + where to find the files. When you run + make in the top-level &yaz; directory, + everything you need to create your server is to link with the + lib/libyaz.la library. - Your main() Routine + Your main() Routine As mentioned, your main() routine can be quite brief. @@ -128,9 +136,9 @@ - int statserv_main(int argc, char **argv, - bend_initresult *(*bend_init)(bend_initrequest *r), - void (*bend_close)(void *handle)); +int statserv_main(int argc, char **argv, + bend_initresult *(*bend_init)(bend_initrequest *r), + void (*bend_close)(void *handle)); @@ -146,95 +154,115 @@ according to the parameters given. When connection requests are received, the event handler will typically fork() and create a sub-process to handle a new connection. - Alternatively the server may be setup to create threads for each connection. + Alternatively the server may be setup to create threads for each + connection. If you do use global variables and forking, you should be aware, then, that these cannot be shared between associations, unless you explicitly disable forking by command line parameters. - + The server provides a mechanism for controlling some of its behavior without using command-line options. The function - statserv_options_block *statserv_getcontrol(void); +statserv_options_block *statserv_getcontrol(void); - Will return a pointer to a struct statserv_options_block + will return a pointer to a struct statserv_options_block describing the current default settings of the server. The structure contains these elements: - + - int dynamic + + int dynamic A boolean value, which determines whether the server will fork on each incoming request (TRUE), or not (FALSE). Default is TRUE. This flag is only read by UNIX-based servers (WIN32 based servers doesn't fork). - - int threads + + + int threads A boolean value, which determines whether the server will create a thread on each incoming request (TRUE), or not (FALSE). - Default is FALSE. This flag is only read by UNIX-based servers that offer - POSIX Threads support. WIN32-based servers always operate in threaded mode. + Default is FALSE. This flag is only read by UNIX-based servers + that offer POSIX Threads support. + WIN32-based servers always operate in threaded mode. - - int inetd + + + int inetd A boolean value, which determines whether the server will operates under a UNIX INET daemon (inetd). Default is FALSE. - - int loglevel + + + int loglevel Set this by ORing the constants defined in include/yaz/yaz-log.h. - - char logfile[ODR_MAXNAME+1] + + + char logfile[ODR_MAXNAME+1] File for diagnostic output ("": stderr). - char apdufile[ODR_MAXNAME+1] + + + char apdufile[ODR_MAXNAME+1] - Name of file for logging incoming and outgoing APDUs ("": don't - log APDUs, "-": stderr). + Name of file for logging incoming and outgoing APDUs + ("": don't log APDUs, "-": + stderr). - char default_listen[1024] + + char default_listen[1024] Same form as the command-line specification of listener address. "": no default listener address. Default is to listen at "tcp:@:9999". You can only specify one default listener address in this fashion. - enum oid_proto default_proto; - Either PROTO_SR or - PROTO_Z3950. Default is PROTO_Z39_50. + + enum oid_proto default_proto; + Either PROTO_Z3950 or + PROTO_SR. + Default is PROTO_Z39_50. - int idle_timeout; - Maximum session idletime, in minutes. Zero indicates - no (infinite) timeout. Default is 120 minutes. + + + int idle_timeout; + Maximum session idle-time, in minutes. Zero indicates + no (infinite) timeout. Default is 15 minutes. - - int maxrecordsize; + + + int maxrecordsize; Maximum permissible record (message) size. Default - is 1Mb. This amount of memory will only be allocated if a client requests a - very large amount of records in one operation (or a big record). Set it - to a lower number - if you are worried about resource consumption on your host system. + is 1Mb. This amount of memory will only be allocated if a + client requests a very large amount of records in one operation + (or a big record). + Set it to a lower number if you are worried about resource + consumption on your host system. - char configname[ODR_MAXNAME+1] + + char configname[ODR_MAXNAME+1] Passed to the backend when a new connection is received. - char setuid[ODR_MAXNAME+1] + + char setuid[ODR_MAXNAME+1] Set user id to the user specified, after binding the listener addresses. - - - void (*bend_start)(struct statserv_options_block *p) + + + void (*bend_start)(struct statserv_options_block *p) + Pointer to function which is called after the command line options have been parsed - but before the server starts listening. @@ -246,17 +274,19 @@ When the server operates as an NT service this handler is called whenever the service is started. - - - void (*bend_stop)(struct statserv_options_block *p) - Pointer to function which is called whenver the server + + + void (*bend_stop)(struct statserv_options_block *p) + + Pointer to function which is called whenever the server has stopped listening for incoming connections. This function pointer has a default value of NULL in which case it isn't called. When the server operates as an NT service this handler is called whenever the service is stopped. - void *handle + + void *handle User defined pointer (default value NULL). This is a per-server handle that can be used to specify "user-data". Do not confuse this with the session-handle as returned by bend_init. @@ -270,9 +300,9 @@ a static area. You are allowed to change the contents of the structure, but the changes will not take effect before you call - + - void statserv_setcontrol(statserv_options_block *block); +void statserv_setcontrol(statserv_options_block *block); @@ -283,7 +313,7 @@ - The Backend Functions + The Backend Functions For each service of the protocol, the backend interface declares one or @@ -294,7 +324,7 @@ Init - bend_initresult (*bend_init)(bend_initrequest *r); +bend_initresult (*bend_init)(bend_initrequest *r); @@ -304,6 +334,13 @@ bend_init handler is passed in the call to statserv_start. + + + This handler is also called when operating in SRW/SRU mode - when + a connection has been made (even though SRW/SRU does not offer + this service). + + Unlike previous versions of YAZ, the bend_init also serves as a handler that defines the Z39.50 services that the backend @@ -323,6 +360,7 @@ typedef struct bend_initrequest Z_ReferenceId *referenceId;/* reference ID */ char *peer_name; /* dns host of peer (client) */ + char *implementation_id; char *implementation_name; char *implementation_version; int (*bend_sort) (void *handle, bend_sort_rr *rr); @@ -333,6 +371,13 @@ typedef struct bend_initrequest int (*bend_delete)(void *handle, bend_delete_rr *rr); int (*bend_scan)(void *handle, bend_scan_rr *rr); int (*bend_segment)(void *handle, bend_segment_rr *rr); + + ODR decode; /* decoding stream */ + /* character set and language negotiation - see include/yaz/z-charneg.h */ + Z_CharSetandLanguageNegotiation *charneg_request; + Z_External *charneg_response; + int (*bend_srw_explain)(void *handle, bend_explain_rr *rr); + int (*bend_srw_scan)(void *handle, bend_scan_rr *rr); } bend_initrequest; typedef struct bend_initresult @@ -375,8 +420,10 @@ typedef struct bend_initresult The members peer_name, + implementation_id, implementation_name and - implementation_version holds DNS of client, name + implementation_version holds + DNS of client, ID of implementor, name of client (Z39.50) implementation - and version. @@ -391,7 +438,7 @@ typedef struct bend_initresult Search and retrieve We now describe the handlers that are required to support search - - and retrieve. You must support two functions - one for seearch - and one + and retrieve. You must support two functions - one for search - and one for fetch (retrieval of one record). If desirable you can provide a third handler which is called when a present request is received which allows you to optimize retrieval of multiple-records. @@ -417,13 +464,13 @@ typedef struct { int hits; /* number of hits */ int errcode; /* 0==OK */ char *errstring; /* system error string or NULL */ + Z_OtherInformation *search_info; } bend_search_rr; - The bend_search handler is a fairly close - approximation of a protocol Search Request - and Response PDUs + approximation of a protocol Z39.50 Search Request - and Response PDUs The setname is the resultSetName from the protocol. You are required to establish a mapping between the set name and whatever your backend database likes to use. @@ -431,8 +478,8 @@ typedef struct { corresponding to the resultSetIndicator field in the protocol. num_bases/basenames is a length of/array of character pointers to the database names provided by the client. - The query is the full query structure as defined in the - protocol ASN.1 specification. + The query is the full query structure as defined in + the protocol ASN.1 specification. It can be either of the possible query types, and it's up to you to determine if you can handle the provided query type. Rather than reproduce the C interface here, we'll refer you to the @@ -454,7 +501,24 @@ typedef struct { defined error codes, and the suggested uses of the addinfo field. + + The bend_search handler is also called when + the frontend server receives a SRW/SRU SearchRetrieveRequest. + For SRW/SRU, a CQL query is usually provided by the client. + The CQL query is available as part of Z_Query + structure (note that CQL is now part of Z39.50 via an external). + To support CQL in existing implementations that only do Type-1, + we refer to the CQL-to-PQF tool described + here. + + + To maintain backwards compatibility, the frontend server + of yaz always assume that error codes are BIB-1 diagnostics. + For SRW/SRU operation, a Bib-1 diagnostic code is mapped to + SRW/SRU diagnostic. + + int (*bend_fetch) (void *handle, bend_fetch_rr *rr); @@ -477,27 +541,38 @@ typedef struct bend_fetch_rr { int errcode; /* 0==success */ char *errstring; /* system error string or NULL */ int surrogate_flag; /* surrogate diagnostic */ + char *schema; /* string record schema input/output */ } bend_fetch_rr; The frontend server calls the bend_fetch handler - when it needs database records to fulfill a Search Request or a Present - Request. + when it needs database records to fulfill a Z39.50 Search Request, a + Z39.50 Present Request or a SRW SearchRetrieveRequest. The setname is simply the name of the result set that holds the reference to the desired record. The number is the offset into the set (with 1 being the first record in the set). The format field - is the record format requested by the client (See section - Object Identifiers). The value - VAL_NONE indicates that the client did not - request a specific format. The stream argument + is the record format requested by the client (See + ). + The value VAL_NONE indicates that the client did + not request a specific format. The stream argument is an &odr; stream which should be used for allocating space for structured data records. The stream will be reset when all records have been assembled, and the response package has been transmitted. - For unstructured data, the backend is responsible for maintaining a static - or dynamic buffer for the record between calls. + For unstructured data, the backend is responsible for maintaining a + static or dynamic buffer for the record between calls. + + + + If a SRW/SRU SearchRetrieveRequest is received by the frontend server, + the referenceId is NULL and the + request_format (transfer syntax) is XML (OID name + VAL_TEXT_XML). + The schema for SRW/SRU is stored in both the + Z_RecordComposition + structure and schema (simple string). @@ -505,7 +580,7 @@ typedef struct bend_fetch_rr { database that holds the record. len is the length of the record returned, in bytes, and record is a pointer to the record. - Last_in_set should be nonzero only if the record + last_in_set should be nonzero only if the record returned is the last one in the given result set. errcode and errstring, if given, will be interpreted as a global error pertaining to the @@ -552,7 +627,7 @@ typedef struct { oid_value format; /* One of the CLASS_RECSYN members */ Z_ReferenceId *referenceId;/* reference ID */ Z_RecordComposition *comp; /* Formatting instructions */ - ODR stream; /* encoding stream - memory source if required */ + ODR stream; /* encoding stream */ ODR print; /* printing stream */ bend_request request; bend_association association; @@ -565,7 +640,8 @@ typedef struct { The bend_present handler is called when - the server receives a Present Request. The setname, + the server receives a Z39.50 Present Request. + The setname, start and number is the name of the result set - start position - and number of records to be retrieved respectively. format and @@ -583,7 +659,7 @@ typedef struct { Delete - For backends that supports delete of a result set only one handler + For back-ends that supports delete of a result set only one handler must be defined. @@ -604,8 +680,8 @@ typedef struct bend_delete_rr { - The delete set function definition is rather primitive, mostly because we - have had no practical need for it as of yet. If someone wants + The delete set function definition is rather primitive, mostly because + we have had no practical need for it as of yet. If someone wants to provide a full delete service, we'd be happy to add the extra parameters that are required. Are there clients out there that will actually delete sets they no longer need? @@ -630,7 +706,7 @@ typedef enum { } bend_scan_status; typedef struct bend_scan_rr { - int num_bases; /* number of elements in databaselist */ + int num_bases; /* number of elements in database list */ char **basenames; /* databases to search */ oid_value attributeset; Z_ReferenceId *referenceId; /* reference ID */ @@ -646,178 +722,124 @@ typedef struct bend_scan_rr { bend_scan_status status; int errcode; char *errstring; + char *scanClause; /* CQL scan clause */ } bend_scan_rr; + + This backend server handles both Z39.50 scan + and SRW/SRU scan. In order for a + handler to distinguish between SRW/SRU (CQL) scan + Z39.50 Scan , it must check for a non-NULL value of + scanClause. + + + + if designed today, it would be a choice using a union or similar, + but that would break binary compatibility with existing servers. + + - Application Invocation + Application Invocation The finished application has the following invocation syntax (by way of statserv_main()): - - appname [-szSiTu -a apdufile -l logfile -v loglevel -c config] - [listener ...] - - + &gfs-synopsis; + - The options are - - - - -a file - - Specify a file for dumping PDUs (for diagnostic purposes). - The special name "-" sends output to - stderr. - - - -S - - Don't fork or make threads on connection requests. This is good for - debugging, but not recommended for real operation: Although the - server is asynchronous and non-blocking, it can be nice to keep - a software malfunction (okay then, a crash) from affecting all - current users. - - - -T - - Operate the server in threaded mode. The server creates a thread - for each connection rather than a fork a process. Only available - on UNIX systems that offers POSIX threads. - - - -s - - Use the SR protocol (obsolete). - - - -z - - Use the Z39.50 protocol (default). These two options complement - each other. You can use both multiple times on the same command - line, between listener-specifications (see below). This way, you - can set up the server to listen for connections in both protocols - concurrently, on different local ports. - - - -l file - The logfile. - + The options are: - -c config - A user option that serves as a specifier for some - sort of configuration, e.g. a filename. - The argument to this option is transferred to member - confignameof the - statserv_options_block. - + &gfs-options; - -v level - - The log level. Use a comma-separated list of members of the set - {fatal,debug,warn,log,all,none}. - - - -u userid - - Set user ID. Sets the real UID of the server process to that of the - given user. It's useful if you aren't comfortable with having the - server run as root, but you need to start it as such to bind a - privileged port. - - - -w dir - - Working directory. - - - -i - - Use this when running from the inetd server. - - - -t minutes - - Idle session timeout, in minutes. - - - -k size - - Maximum record size/message size, in kilobytes. - - - - + A listener specification consists of a transport mode followed by a colon (:) followed by a listener address. The transport mode is - either osi or tcp. + either tcp, unix: or + ssl. - + - For TCP, an address has the form + For TCP and SSL, an address has the form hostname | IP-number [: portnumber] - + The port number defaults to 210 (standard Z39.50 port). - For osi, the address form is - - - - [t-selector /] hostname | IP-number [: portnumber] - - - - The transport selector is given as a string of hex digits (with an even - number of digits). The default port number is 102 (RFC1006 port). - - - - Examples - - - - tcp:dranet.dra.com - - osi:0402/dbserver.osiworld.com:3000 - - - - In both cases, the special hostname "@" is mapped to - the address INADDR_ANY, which causes the server to listen on any local - interface. To start the server listening on the registered ports for - Z39.50 and SR over OSI/RFC1006, and to drop root privileges once the - ports are bound, execute the server like this (from a root shell): + For UNIX, the address is the filename of socket. - - my-server -u daemon tcp:@ -s osi:@ - - - You can replace daemon with another user, eg. your - own account, or a dedicated IR server account. - my-server should be the name of your - server application. You can test the procedure with the - yaz-ztest application. + For TCP/IP and SSL, the special hostname @ + (at sign) is mapped to the address INADDR_ANY, + which causes the server to listen on any local interface. + Running the GFS on Unix + + Assuming the server application appname is + started as root, the following will make it listen on port 210. + The server will change identity to nobody + and write its log to /var/log/app.log. + + appname -l /var/log/app.log -u nobody tcp:@:210 + + + + The server will accept Z39.50 requests and offer SRW/SRU service + on port 210. + + + Setting up Apache as SRW/SRU Frontend + + If you use Apache + as your public web server and want to offer HTTP port 80 + access to the YAZ server on 210, you can use the + + ProxyPass + directive. + If you have virtual host + srw.mydomain you can use the following directives + in Apache's httpd.conf: + + <VirtualHost *> + ErrorLog /home/srw/logs/error_log + TransferLog /home/srw/logs/access_log + ProxyPass / http://srw.mydomain:210/ + </VirtualHost> + + + + The above for the Apache 1.3 series. + + + Running a server with local access only + + Servers that is only being accessed from the local host should listen + on UNIX file socket rather than a Internet socket. To listen on + /tmp/mysocket start the server as follows: + + appname tcp:/tmp/mysocket + + + + + Virtual Hosts + &gfs-virtual; - +