Honor position attribute, i.e. allow first-in-field search. To

[idzebra-moved-to-github.git] / doc / administration.xml

diff --git a/doc/administration.xml b/doc/administration.xml
index 7303d30..c4afea4 100644 (file)
--- a/doc/administration.xml
+++ b/doc/administration.xml
@@ -1,5 +1,5 @@
 <chapter id="administration">
 <chapter id="administration">

- <!-- $Id: administration.xml,v 1.39 2006-06-13 13:45:08 marc Exp $ -->
+ <!-- $Id: administration.xml,v 1.44 2006-09-05 12:01:31 adam Exp $ -->

  <title>Administrating Zebra</title>
  <!-- ### It's a bit daft that this chapter (which describes half of
           the configuration-file formats) is separated from
  <title>Administrating Zebra</title>
  <!-- ### It's a bit daft that this chapter (which describes half of
           the configuration-file formats) is separated from

@@ -94,7 +94,7 @@
   
  </sect1>
  
   
  </sect1>
  

- <sect1 id="configuration-file">
+ <sect1 id="zebra-cfg">

   <title>The Zebra Configuration File</title>
   
   <para>
   <title>The Zebra Configuration File</title>
   
   <para>

@@ -356,7 +356,7 @@
        Specifies a file with description of user accounts for Zebra.
        File format is similar to that used by the passwd directive except
        that the password are encrypted. Use Apache's htpasswd or similar
        Specifies a file with description of user accounts for Zebra.
        File format is similar to that used by the passwd directive except
        that the password are encrypted. Use Apache's htpasswd or similar

-       for maintenanace.
+       for maintenance.

       </para>
      </listitem>
     </varlistentry>
       </para>
      </listitem>
     </varlistentry>

@@ -370,7 +370,7 @@
        to access Zebra via the passwd system. There are two kinds
        of permissions currently: read (r) and write(w). By default
        users not listed in a permission directive are given the read
        to access Zebra via the passwd system. There are two kinds
        of permissions currently: read (r) and write(w). By default
        users not listed in a permission directive are given the read

-       priviledge. To specify permissions for a user with no
+       privilege. To specify permissions for a user with no

        username, or Z39.50 anonymous style use
        <literal>anonymous</literal>. The permstring consists of
        a sequence of characters. Include character <literal>w</literal>
        username, or Z39.50 anonymous style use
        <literal>anonymous</literal>. The permstring consists of
        a sequence of characters. Include character <literal>w</literal>

@@ -669,7 +669,7 @@
   </para>
   
   <para>
   </para>
   
   <para>

-   (see <xref linkend="record-model-grs"/>
+   (see <xref linkend="grs"/>

     for details of how the mapping between elements of your records and
     searchable attributes is established).
   </para>
     for details of how the mapping between elements of your records and
     searchable attributes is established).
   </para>

@@ -757,7 +757,7 @@
  <sect1 id="shadow-registers">
   <title>Safe Updating - Using Shadow Registers</title>
   
  <sect1 id="shadow-registers">
   <title>Safe Updating - Using Shadow Registers</title>
   

-  <sect2>
+  <sect2 id="shadow-registers-description">

    <title>Description</title>
    
    <para>
    <title>Description</title>
    
    <para>

@@ -811,7 +811,7 @@
    
   </sect2>
   
    
   </sect2>
   

-  <sect2>
+  <sect2 id="shadow-registers-how-to-use">

    <title>How to Use Shadow Register Files</title>
    
    <para>
    <title>How to Use Shadow Register Files</title>
    
    <para>

@@ -925,7 +925,7 @@
  <sect1 id="administration-ranking">
   <title>Relevance Ranking and Sorting of Result Sets</title>
 
  <sect1 id="administration-ranking">
   <title>Relevance Ranking and Sorting of Result Sets</title>
 

-  <sect2>
+  <sect2 id="administration-overview">

    <title>Overview</title>
    <para>
     The default ordering of a result set is left up to the server,
    <title>Overview</title>
    <para>
     The default ordering of a result set is left up to the server,

@@ -1288,7 +1288,6 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
     with or without static ranking enabled.
     </para>
  
     with or without static ranking enabled.
     </para>
  

-    </sect3>

 
     <!--
     <sect3 id="administration-ranking-dynamic-rank1">
 
     <!--
     <sect3 id="administration-ranking-dynamic-rank1">

@@ -1308,7 +1307,6 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
    </para>
     </sect3>
     -->
    </para>
     </sect3>
     -->

-

  
    <warning>
      <para>
  
    <warning>
      <para>

@@ -1332,6 +1330,7 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
     mitaylor2microsoft.com
    -->
 
     mitaylor2microsoft.com
    -->
 

+    </sect3>

 
     <sect3 id="administration-ranking-dynamic-cql">
      <title>Dynamically ranking CQL queries</title>
 
     <sect3 id="administration-ranking-dynamic-cql">
      <title>Dynamically ranking CQL queries</title>

@@ -1436,26 +1435,216 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
  <sect1 id="administration-extended-services">
   <title>Extended Services: Remote Insert, Update and Delete</title>
   
  <sect1 id="administration-extended-services">
   <title>Extended Services: Remote Insert, Update and Delete</title>
   

+   <note>
+    <para>
+     Extended services are only supported when accessing the Zebra
+     server using the <ulink url="&url.z39.50;">Z39.50</ulink>
+     protocol. The <ulink url="&url.sru;">SRU</ulink> protocol does
+     not support extended services.
+    </para>
+   </note>
+   

   <para>
     The extended services are not enabled by default in zebra - due to the
   <para>
     The extended services are not enabled by default in zebra - due to the

-    fact that they modify the system.
-    In order to allow anybody to update, use
-    <screen>
-    perm.anonymous: rw
-    </screen>
+    fact that they modify the system. Zebra can be configured
+    to allow anybody to
+    search, and to allow only updates for a particular admin user

     in the main zebra configuration file <filename>zebra.cfg</filename>.
     in the main zebra configuration file <filename>zebra.cfg</filename>.

-    Or, even better, allow only updates for a particular admin user. For
-    user <literal>admin</literal>, you could use:
+    For user <literal>admin</literal>, you could use:

     <screen>
     <screen>

+     perm.anonymous: r

      perm.admin: rw
      passwd: passwordfile
     </screen>
      perm.admin: rw
      passwd: passwordfile
     </screen>

-    And in <filename>passwordfile</filename>, specify users and
-    passwords as colon seperated strings:
+    And in the password file 
+    <filename>passwordfile</filename>, you have to specify users and
+    encrypted passwords as colon separated strings. 
+    Use a tool like <filename>htpasswd</filename> 
+    to maintain the encrypted passwords. 

     <screen> 
      admin:secret
     <screen> 
      admin:secret

-    </screen> 
+    </screen>
+    It is essential to configure  Zebra to store records internally, 
+    and to support
+    modifications and deletion of records:
+    <screen>
+     storeData: 1
+     storeKeys: 1
+    </screen>
+    The general record type should be set to any record filter which
+    is able to parse XML records, you may use any of the two
+    declarations (but not both simultaneously!)
+    <screen>    
+     recordType: grs.xml
+     # recordType: alvis.filter_alvis_config.xml
+    </screen>
+    To enable transaction safe shadow indexing,
+    which is extra important for this kind of operation, set
+    <screen>
+     shadow: directoryname: size (e.g. 1000M)
+    </screen>
+   </para>
+   <note>
+    <para>
+     It is not possible to carry information about record types or
+     similar to Zebra when using extended services, due to
+     limitations of the <ulink url="&url.z39.50;">Z39.50</ulink>
+     protocol. Therefore, indexing filters can not be chosen on a
+     per-record basis. One and only one general XML indexing filter
+     must be defined.  
+     <!-- but because it is represented as an OID, we would need some
+     form of proprietary mapping scheme between record type strings and
+     OIDs. -->
+     <!--
+     However, as a minimum, it would be extremely useful to enable
+     people to use MARC21, assuming grs.marcxml.marc21 as a record
+     type.  
+     -->
+    </para>
+   </note>
+
+
+   <sect2 id="administration-extended-services-z3950">
+    <title>Extended services in the Z39.50 protocol</title>
+
+    <para>
+     The <ulink url="&url.z39.50;">Z39.50</ulink> standard allows
+     servers to accept special binary <emphasis>extended services</emphasis>
+     protocol packages, which may be used to insert, update and delete
+     records into servers. These carry  control and update
+     information to the servers, which are encoded in seven package fields: 
+    </para>
+
+    <table id="administration-extended-services-z3950-table" frame="top">
+     <title>Extended services Z39.50 Package Fields</title>
+      <tgroup cols="3">
+       <thead>
+       <row>
+         <entry>Parameter</entry>
+         <entry>Value</entry>
+         <entry>Notes</entry>
+        </row>
+      </thead>
+       <tbody>
+        <row>
+         <entry><literal>type</literal></entry>
+         <entry><literal>'update'</literal></entry>
+         <entry>Must be set to trigger extended services</entry>
+        </row>
+        <row>
+         <entry><literal>action</literal></entry>
+         <entry><literal>string</literal></entry>
+        <entry>
+         Extended service action type with 
+         one of four possible values: <literal>recordInsert</literal>,
+         <literal>recordReplace</literal>,
+         <literal>recordDelete</literal>,
+         and <literal>specialUpdate</literal>
+        </entry>
+        </row>
+        <row>
+         <entry><literal>record</literal></entry>
+         <entry><literal>XML string</literal></entry>
+         <entry>An XML formatted string containing the record</entry>
+        </row>
+        <row>
+         <entry><literal>syntax</literal></entry>
+         <entry><literal>'xml'</literal></entry>
+         <entry>Only XML record syntax is supported</entry>
+        </row>
+        <row>
+         <entry><literal>recordIdOpaque</literal></entry>
+         <entry><literal>string</literal></entry>
+         <entry>
+         Optional  client-supplied, opaque record
+         identifier used under insert operations.
+        </entry>
+        </row>
+        <row>
+         <entry><literal>recordIdNumber </literal></entry>
+         <entry><literal>positive number</literal></entry>
+         <entry>Zebra's internal system number, only for update
+         actions.
+        </entry>
+        </row>
+        <row>
+         <entry><literal>databaseName</literal></entry>
+         <entry><literal>database identifier</literal></entry>
+        <entry>
+         The name of the database to which the extended services should be 
+         applied.
+        </entry>
+        </row>
+      </tbody>
+      </tgroup>
+     </table>
+
+
+   <para>
+    The <literal>action</literal> parameter can be any of 
+    <literal>recordInsert</literal> (will fail if the record already exists),
+    <literal>recordReplace</literal> (will fail if the record does not exist),
+    <literal>recordDelete</literal> (will fail if the record does not
+       exist), and
+    <literal>specialUpdate</literal> (will insert or update the record
+       as needed).

    </para>
    </para>

+
+    <para>
+     During a  <literal>recordInsert</literal> action, the
+     usual rules for internal record ID generation apply, unless an
+     optional <literal>recordIdNumber</literal> Zebra internal ID or a
+    <literal>recordIdOpaque</literal> string identifier is assigned. 
+     The default ID generation is
+     configured using the <literal>recordId:</literal> from
+     <filename>zebra.cfg</filename>.     
+    </para>
+
+   <para>
+    The actions <literal>recordReplace</literal> or
+    <literal>recordDelete</literal> need specification of the additional 
+    <literal>recordIdNumber</literal> parameter, which must be an
+    existing Zebra internal system ID number, or the optional 
+     <literal>recordIdOpaque</literal> string parameter.
+    </para>
+
+    <para>
+     When retrieving existing
+     records indexed with GRS indexing filters, the Zebra internal 
+     ID number is returned in the field
+    <literal>/*/id:idzebra/localnumber</literal> in the namespace
+    <literal>xmlns:id="http://www.indexdata.dk/zebra/"</literal>,
+    where it can be picked up for later record updates or deletes. 
+    </para>
+    <para>
+     Records indexed with the <literal>alvis</literal> filter
+     have similar means to discover the internal Zebra ID.
+    </para>
+ 
+   <para>
+     The <literal>recordIdOpaque</literal> string parameter
+     is an client-supplied, opaque record
+     identifier, which may be  used under 
+     insert, update and delete operations. The
+     client software is responsible for assigning these to
+     records.      This identifier will
+     replace zebra's own automagic identifier generation with a unique
+     mapping from <literal>recordIdOpaque</literal> to the 
+     Zebra internal <literal>recordIdNumber</literal>.
+     <emphasis>The opaque <literal>recordIdOpaque</literal> string
+     identifiers
+      are not visible in retrieval records, nor are
+      searchable, so the value of this parameter is
+      questionable. It serves mostly as a convenient mapping from
+      application domain string identifiers to Zebra internal ID's.
+     </emphasis> 
+    </para>
+   </sect2>
+
+   
+ <sect2 id="administration-extended-services-yaz-client">
+  <title>Extended services from yaz-client</title>
+

    <para>
     We can now start a yaz-client admin session and create a database:
    <screen>
    <para>
     We can now start a yaz-client admin session and create a database:
    <screen>

@@ -1469,14 +1658,11 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
     from example/gils/records) and index it:
    <screen>  
     <![CDATA[
     from example/gils/records) and index it:
    <screen>  
     <![CDATA[

-     Z> update insert 1 esdd0006.grs
+     Z> update insert id1234 esdd0006.grs

      ]]>
    </screen>
      ]]>
    </screen>

-    The 3rd parameter - <literal>1</literal> here -
-      is the opaque record ID from <literal>Ext update</literal>.
-      It a record ID that <emphasis>we</emphasis> assign to the record
-    in question. If we do not 
-    assign one, the usual rules for match apply (recordId: from zebra.cfg).
+    The 3rd parameter - <literal>id1234</literal> here -
+      is the  <literal>recordIdOpaque</literal> package field.

    </para>
    <para>
     Actually, we should have a way to specify "no opaque record id" for
    </para>
    <para>
     Actually, we should have a way to specify "no opaque record id" for

@@ -1498,10 +1684,11 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
     </screen>
    </para>
    <para>
     </screen>
    </para>
    <para>

-    Let's delete the beast:
+     Let's delete the beast, using the same 
+     <literal>recordIdOpaque</literal> string parameter:

     <screen>
     <![CDATA[
     <screen>
     <![CDATA[

-     Z> update delete 1
+     Z> update delete id1234

      No last record (update ignored)
      Z> update delete 1 esdd0006.grs
      Got extended services response
      No last record (update ignored)
      Z> update delete 1 esdd0006.grs
      Got extended services response

@@ -1530,8 +1717,14 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
      after each update session in order write your changes from the
      shadow to the life register space.
    </para>
      after each update session in order write your changes from the
      shadow to the life register space.
    </para>

+ </sect2>
+
+  
+ <sect2 id="administration-extended-services-yaz-php">
+  <title>Extended services from yaz-php</title>
+

    <para>
    <para>

-    Extended services are also available from the YAZ client layer. An
+    Extended services are also available from the YAZ PHP client layer. An

     example of an YAZ-PHP extended service transaction is given here:
     <screen>
     <![CDATA[
     example of an YAZ-PHP extended service transaction is given here:
     <screen>
     <![CDATA[

@@ -1551,51 +1744,10 @@ where g = rset_count(terms[i]->rset) is the count of all documents in this speci
        echo "$error";
      ]]>
     </screen>  
        echo "$error";
      ]]>
     </screen>  

-    The <literal>action</literal> parameter can be any of 
-    <literal>recordInsert</literal> (will fail if the record already exists),
-    <literal>recordReplace</literal> (will fail if the record does not exist),
-    <literal>recordDelete</literal> (will fail if the record does not
-       exist), and
-    <literal>specialUpdate</literal> (will insert or update the record
-       as needed).
-   </para>
-   <para>
-    If a record is inserted
-    using the action  <literal>recordInsert</literal> 
-    one can specify the optional
-    <literal>recordIdOpaque</literal> parameter, which is a
-    client-supplied, opaque record identifier. This identifier will
-    replace zebra's own automagic identifier generation.  
-   </para>
-   <para>
-    When using the action <literal>recordReplace</literal> or
-    <literal>recordDelete</literal>, one must specify the additional 
-    <literal>recordIdNumber</literal> parameter, which must be an
-    existing Zebra internal system ID number. When retrieving existing
-    records, the ID number is returned in the field
-    <literal>/*/id:idzebra/localnumber</literal> in the namespace
-    <literal>xmlns:id="http://www.indexdata.dk/zebra/"</literal>,
-    where it can be picked up for later record updates or deletes. 
-   </para>
- </sect1>
-
-
-  <sect1 id="gfs-config">
-   <title>YAZ Frontend Virtual Hosts</title>
-    <para>
-     <command>zebrasrv</command> uses the YAZ server frontend and does
-     support multiple virtual servers behind multiple listening sockets.

     </para>
     </para>

-    &zebrasrv-virtual;
- 
-   <para>
-    Section "Virtual Hosts" in the YAZ manual.
-    <filename>http://www.indexdata.dk/yaz/doc/server.vhosts.tkl</filename>
-   </para>
+    </sect2>

  </sect1>
 
  </sect1>
 

-
- 

 </chapter>
 
  <!-- Keep this comment at the end of the file
 </chapter>
 
  <!-- Keep this comment at the end of the file