X-Git-Url: http://git.indexdata.com/?p=idzebra-moved-to-github.git;a=blobdiff_plain;f=doc%2Frecordmodel-grs.xml;h=853410a0f203c748cb88382960d5dbd51fd7fee4;hp=ffa39f42e6e870eb48cff420457ff0246f9e0398;hb=dcda88860b03641b6900d43135ca769f005105e8;hpb=47054fae00306e75212a26ee5305f00032c99001 diff --git a/doc/recordmodel-grs.xml b/doc/recordmodel-grs.xml index ffa39f4..853410a 100644 --- a/doc/recordmodel-grs.xml +++ b/doc/recordmodel-grs.xml @@ -1,7 +1,13 @@ - - - GRS Record Model and Filter Modules - + + &acro.grs1; Record Model and Filter Modules + + + + The functionality of this record model has been improved and + replaced by the DOM &acro.xml; record model. See + . + + The record model described in this chapter applies to the fundamental, @@ -11,8 +17,8 @@ - - GRS Record Filters +
+ &acro.grs1; Record Filters Many basic subtypes of the grs type are currently available: @@ -21,128 +27,124 @@ - grs.sgml + grs.sgml This is the canonical input format described . It is using - simple SGML-like syntax. - - - grs.marc + grs.marc.type - This allows Zebra to read - records in the ISO2709 (MARC) encoding standard. - + which describes the specific &acro.marc; structure of the input record as + well as the indexing rules. + + The grs.marc uses an internal representation + which is not &acro.xml; conformant. In particular &acro.marc; tags are + presented as elements with the same name. And &acro.xml; elements + may not start with digits. Therefore this filter is only + suitable for systems returning &acro.grs1; and &acro.marc; records. For &acro.xml; + use grs.marcxml filter instead (see below). - The loadable grs.marc filter module - is packaged in the GNU/Debian package - libidzebra1.4-mod-grs-marc - + The loadable grs.marc filter module + is packaged in the GNU/Debian package + libidzebra2.0-mod-grs-marc + - grs.marcxml + grs.marcxml.type - This allows Zebra to read - records in the ISO2709??? (MARCXML) encoding standard. + This allows &zebra; to read ISO2709 encoded records. + Last parameter type names the + .abs file (see below) + which describes the specific &acro.marc; structure of the input record as + well as the indexing rules. - The loadable grs.marcxml filter module - is also contained in the GNU/Debian package - libidzebra1.4-mod-grs-marc - - - - - grs.danbib - - - The grs.danbib filter parses DanBib - records, a danish MARC record variant called DANMARC. - DanBib is the Danish Union Catalogue hosted by the - Danish Bibliographic Centre (DBC). + The internal representation for grs.marcxml + is the same as for &acro.marcxml;. + It slightly more complicated to work with than + grs.marc but &acro.xml; conformant. - The loadable grs.danbib filter module - is packages in the GNU/Debian package - libidzebra1.4-mod-grs-danbib. + + The loadable grs.marcxml filter module + is also contained in the GNU/Debian package + libidzebra2.0-mod-grs-marc - grs.xml + grs.xml - This filter reads XML records and uses Expat to - parse them and convert them into IDZebra's internal + This filter reads &acro.xml; records and uses + Expat to + parse them and convert them into ID&zebra;'s internal grs record model. - Only one record per file - is supported. The filter is only available if Zebra/YAZ - is compiled with EXPAT support. + Only one record per file is supported, due to the fact &acro.xml; does + not allow two documents to "follow" each other (there is no way + to know when a document is finished). + This filter is only available if &zebra; is compiled with EXPAT support. - The loadable grs.xml filter module - is packagged in the GNU/Debian package - libidzebra1.4-mod-grs-xml - + The loadable grs.xml filter module + is packaged in the GNU/Debian package + libidzebra2.0-mod-grs-xml + - grs.regx + grs.regx.filter This enables a user-supplied Regular Expressions input - filter described in - . + filter described in . - The loadable grs.regx filter module - is packaged in the GNU/Debian package - libidzebra1.4-mod-grs-regx - + The loadable grs.regx filter module + is packaged in the GNU/Debian package + libidzebra2.0-mod-grs-regx + - grs.tcl + grs.tcl.filter - Similar to grs.regx but using Tcl for rules, described in + Similar to grs.regx but using Tcl for rules, described in . - The loadable grs.tcl filter module - is also packaged in the GNU/Debian package - libidzebra1.4-mod-grs-regx - + The loadable grs.tcl filter module + is also packaged in the GNU/Debian package + libidzebra2.0-mod-grs-regx + - - GRS Canonical Input Format +
+ &acro.grs1; Canonical Input Format Although input data can take any form, it is sometimes useful to describe the record processing capabilities of the system in terms of a single, canonical input format that gives access to the full - spectrum of structure and flexibility in the system. In Zebra, this - canonical format is an "SGML-like" syntax. + spectrum of structure and flexibility in the system. In &zebra;, this + canonical format is an "&acro.sgml;-like" syntax. @@ -162,16 +164,16 @@ <Distributor> - <Name> USGS/WRD </Name> - <Organization> USGS/WRD </Organization> - <Street-Address> - U.S. GEOLOGICAL SURVEY, 505 MARQUETTE, NW - </Street-Address> - <City> ALBUQUERQUE </City> - <State> NM </State> - <Zip-Code> 87102 </Zip-Code> - <Country> USA </Country> - <Telephone> (505) 766-5560 </Telephone> + <Name> USGS/WRD </Name> + <Organization> USGS/WRD </Organization> + <Street-Address> + U.S. GEOLOGICAL SURVEY, 505 MARQUETTE, NW + </Street-Address> + <City> ALBUQUERQUE </City> + <State> NM </State> + <Zip-Code> 87102 </Zip-Code> + <Country> USA </Country> + <Telephone> (505) 766-5560 </Telephone> </Distributor> @@ -179,12 +181,12 @@ @@ -207,7 +209,7 @@ structured data element such a Supplier element. - +
Record Root @@ -219,8 +221,8 @@ The following is a GILS record that contains only a single element (strictly speaking, that makes it an illegal GILS record, since the GILS profile includes several mandatory - elements - Zebra does not validate the contents of a record against - the Z39.50 profile, however - it merely attempts to match up elements + elements - &zebra; does not validate the contents of a record against + the &acro.z3950; profile, however - it merely attempts to match up elements of a local representation with the given schema): @@ -228,24 +230,24 @@ <gils> - <title>Zen and the Art of Motorcycle Maintenance</title> + <title>Zen and the Art of Motorcycle Maintenance</title> </gils> - +
- +
Variants - Zebra allows you to provide individual data elements in a number of + &zebra; allows you to provide individual data elements in a number of variant forms. Examples of variant forms are textual data elements which might appear in different languages, and images which may appear in different formats or layouts. - The variant system in Zebra is essentially a representation of - the variant mechanism of Z39.50-1995. + The variant system in &zebra; is essentially a representation of + the variant mechanism of &acro.z3950;-1995. @@ -272,7 +274,7 @@ The available values for the class and type fields are given by the variant set that is associated with the current schema - (see ). + (see ). @@ -325,21 +327,21 @@ The title element above comes in two variants. Both have the IANA body type "text/plain", but one is in English, and the other in - Danish. The client, using the element selection mechanism of Z39.50, + Danish. The client, using the element selection mechanism of &acro.z3950;, can retrieve information about the available variant forms of data elements, or it can select specific variants based on the requirements of the end-user. - +
- +
- - GRS REGX And TCL Input Filters +
+ &acro.grs1; REGX And TCL Input Filters - In order to handle general input formats, Zebra allows the + In order to handle general input formats, &zebra; allows the operator to define filters which read individual records in their native format and produce an internal representation that the system can work with. @@ -357,7 +359,7 @@ type regx, argument filter-filename). - + Generally, an input filter consists of a sequence of rules, where each rule consists of a sequence of expressions, followed by an action. The @@ -365,7 +367,7 @@ and the actions normally contribute to the generation of an internal representation of the record. - + An expression can be either of the following: @@ -374,7 +376,7 @@ - INIT + INIT The action associated with this expression is evaluated @@ -386,7 +388,7 @@ - BEGIN + BEGIN Matches the beginning of the record. It can be used to @@ -397,7 +399,7 @@ - END + END Matches the end of the record - when all of the contents @@ -406,15 +408,20 @@ - /pattern/ + + /reg/ + - Matches a string of characters from the input record. + Matches regular expression pattern reg + from the input record. The operators supported are the same + as for regular expression queries. Refer to + . - BODY + BODY This keyword may only be used between two patterns. @@ -423,7 +430,7 @@ - FINISH + FINISH The expression associated with this pattern is evaluated @@ -460,13 +467,13 @@ data element. The type is one of the following: - + record Begin a new record. The following parameter should be the - name of the schema that describes the structure of the record, eg. + name of the schema that describes the structure of the record, e.g., gils or wais (see below). The begin record call should precede any other use of the begin statement. @@ -561,29 +568,29 @@ /^Subject:/ BODY /$/ { data -element title $1 } /^Date:/ BODY /$/ { data -element lastModified $1 } /\n\n/ BODY END { - begin element bodyOfDisplay - begin variant body iana "text/plain" - data -text $1 - end record + begin element bodyOfDisplay + begin variant body iana "text/plain" + data -text $1 + end record } - If Zebra is compiled with support for Tcl enabled, the statements + If &zebra; is compiled with support for Tcl enabled, the statements described above are supplemented with a complete scripting environment, including control structures (conditional expressions and loop constructs), and powerful string manipulation mechanisms for modifying the elements of a record. - +
- +
- - GRS Internal Record Representation +
+ &acro.grs1; Internal Record Representation When records are manipulated by the system, they're represented in a @@ -597,9 +604,9 @@ - ROOT - TITLE "Zen and the Art of Motorcycle Maintenance" - AUTHOR "Robert Pirsig" + ROOT + TITLE "Zen and the Art of Motorcycle Maintenance" + AUTHOR "Robert Pirsig" @@ -612,11 +619,11 @@ - ROOT - TITLE "Zen and the Art of Motorcycle Maintenance" - AUTHOR - FIRST-NAME "Robert" - SURNAME "Pirsig" + ROOT + TITLE "Zen and the Art of Motorcycle Maintenance" + AUTHOR + FIRST-NAME "Robert" + SURNAME "Pirsig" @@ -633,7 +640,7 @@ different tag path. - +
Tagged Elements @@ -650,9 +657,9 @@ reached from the root of the record). - +
- +
Variants @@ -680,44 +687,44 @@ Which of the two elements are transmitted to the client by the server depends on the specifications provided by the client, if any. - + In practice, each variant node is associated with a triple of class, - type, value, corresponding to the variant mechanism of Z39.50. + type, value, corresponding to the variant mechanism of &acro.z3950;. - - - - + +
+ +
Data Elements - + Data nodes have no children (they are always leaf nodes in the record tree). - + - - - - - - - GRS Record Model Configuration - + +
+ +
+ +
+ &acro.grs1; Record Model Configuration + The following sections describe the configuration files that govern - the internal management of grs records. + the internal management of grs records. The system searches for the files in the directories specified by the profilePath setting in the zebra.cfg file. - +
The Abstract Syntax @@ -728,7 +735,7 @@ @@ -737,7 +744,7 @@ - The object identifier of the Z39.50 schema associated + The object identifier of the &acro.z3950; schema associated with the ARS, so that it can be referred to by the client. @@ -759,7 +766,7 @@ known. - + The variant set which is used in the profile. This provides a @@ -774,7 +781,7 @@ ask for a subset of the data elements contained in a record. Element set names, in the retrieval module, are mapped to element specifications, which contain information equivalent to the - Espec-1 syntax of Z39.50. + Espec-1 syntax of &acro.z3950;. @@ -788,15 +795,15 @@ Possibly, a set of rules describing the mapping of elements to a - MARC representation. + &acro.marc; representation. - + A list of element descriptions (this is the actual ARS of the - schema, in Z39.50 terms), which lists the ways in which the various + schema, in &acro.z3950; terms), which lists the ways in which the various tags can be used and organized hierarchically. @@ -810,9 +817,9 @@ describe the given objects. - +
- +
The Configuration Files @@ -822,7 +829,7 @@ The number of different file types may appear daunting at first, but - each type corresponds fairly clearly to a single aspect of the Z39.50 + each type corresponds fairly clearly to a single aspect of the &acro.z3950; retrieval facilities. Further, the average database administrator, who is simply reusing an existing profile for which tables already exist, shouldn't have to worry too much about the contents of these tables. @@ -840,27 +847,27 @@ file. Some settings are optional (o), while others again are mandatory (m). - - - - + +
+ +
The Abstract Syntax (.abs) Files - + - The name of this file type is slightly misleading in Z39.50 terms, + The name of this file type is slightly misleading in &acro.z3950; terms, since, apart from the actual abstract syntax of the profile, it also includes most of the other definitions that go into a database profile. - + - When a record in the canonical, SGML-like format is read from a file + When a record in the canonical, &acro.sgml;-like format is read from a file or from the database, the first tag of the file should reference the profile that governs the layout of the record. If the first tag of the record is, say, <gils>, the system will look for the profile definition in the file gils.abs. Profile definitions are cached, so they only have to be read once - during the lifespan of the current process. + during the lifespan of the current process. @@ -869,14 +876,14 @@ introduces the profile, and should always be called first thing when introducing a new record. - + The file may contain the following directives: - + - + name symbolic-name @@ -892,7 +899,7 @@ (m) The reference name of the OID for the profile. The reference names can be found in the util - module of YAZ. + module of &yaz;. @@ -938,7 +945,7 @@ (o) Points to a file containing parameters for representing the record contents in the ISO2709 syntax. - Read the description of the MARC representation facility below. + Read the description of the &acro.marc; representation facility below. @@ -954,7 +961,7 @@ - any tags + all tags (o) This directive specifies a list of attributes @@ -974,29 +981,29 @@ (o,r) Adds an element to the abstract record syntax of the schema. The path follows the - syntax which is suggested by the Z39.50 document - that is, a sequence + syntax which is suggested by the &acro.z3950; document - that is, a sequence of tags separated by slashes (/). Each tag is given as a comma-separated pair of tag type and -value surrounded by parenthesis. The name is the name of the element, and the attributes specifies which attributes to use when indexing the element in a comma-separated list. - A ! in place of the attribute name is equivalent to - specifying an attribute name identical to the element name. - A - in place of the attribute name + A ! in place of the attribute name is equivalent + to specifying an attribute name identical to the element name. + A - in place of the attribute name specifies that no indexing is to take place for the given element. The attributes can be qualified with field types to specify which character set should govern the indexing procedure for that field. The same data element may be indexed into several different fields, using different character set definitions. - See the . + See the . The default field type is w for word. - + xelm xpath attributes @@ -1021,8 +1028,8 @@ melm field$subfield attributes - This directive is specifically for MARC-formatted records, - ingested either in the form of MARCXML documents, or in the + This directive is specifically for &acro.marc;-formatted records, + ingested either in the form of &acro.marcxml; documents, or in the ISO2709/Z39.2 format using the grs.marcxml input filter. You can specify indexing rules for any subfield, or you can leave off the $subfield part and specify default rules @@ -1038,11 +1045,11 @@ This directive specifies character encoding for external records. - For records such as XML that specifies encoding within the + For records such as &acro.xml; that specifies encoding within the file via a header this directive is ignored. If neither this directive is given, nor an encoding is set within external records, ISO-8859-1 encoding is assumed. - + @@ -1051,60 +1058,60 @@ If this directive is followed by enable, then extra indexing is performed to allow for XPath-like queries. - If this directive is not specified - equivalent to + If this directive is not specified - equivalent to disable - no extra XPath-indexing is performed. - @@ -1116,8 +1123,8 @@ - Specifies what information, if any, Zebra should - automatically include in retrieval records for the + Specifies what information, if any, &zebra; should + automatically include in retrieval records for the ``system fields'' that it supports. systemTag may be any of the following: @@ -1125,24 +1132,24 @@ rank - An integer indicating the relevance-ranking score - assigned to the record. - + An integer indicating the relevance-ranking score + assigned to the record. + sysno - An automatically generated identifier for the record, - unique within this database. It is represented by the - <localControlNumber> element in - XML and the (1,14) tag in GRS-1. - + An automatically generated identifier for the record, + unique within this database. It is represented by the + <localControlNumber> element in + &acro.xml; and the (1,14) tag in &acro.grs1;. + size - The size, in bytes, of the retrieved record. - + The size, in bytes, of the retrieved record. + @@ -1155,7 +1162,7 @@ - + The mechanism for controlling indexing is not adequate for @@ -1163,7 +1170,7 @@ configuration table eventually. - + The following is an excerpt from the abstract syntax file for the GILS profile. @@ -1195,7 +1202,7 @@ elm (4,1) controlIdentifier Identifier-standard elm (2,6) abstract Abstract elm (4,51) purpose ! - elm (4,52) originator - + elm (4,52) originator - elm (4,53) accessConstraints ! elm (4,54) useConstraints ! elm (4,70) availability - @@ -1208,17 +1215,17 @@ - +
- +
The Attribute Set (.att) Files This file type describes the Use elements of - an attribute set. - It contains the following directives. + an attribute set. + It contains the following directives. - + @@ -1237,7 +1244,7 @@ (m) The reference name of the OID for the attribute set. The reference names can be found in the util - module of YAZ. + module of &yaz;. @@ -1250,7 +1257,7 @@ set. For instance, many new attribute sets are defined as extensions to the bib-1 set. This is an important feature of the retrieval - system of Z39.50, as it ensures the highest possible level of + system of &acro.z3950;, as it ensures the highest possible level of interoperability, as those access points of your database which are derived from the external set (say, bib-1) can be used even by clients who are unaware of the new set. @@ -1266,7 +1273,7 @@ attribute value is stored in the index (unless a local-value is given, in which case this is stored). The name is used to refer to the - attribute from the abstract syntax. + attribute from the abstract syntax. @@ -1294,15 +1301,15 @@ - +
- +
The Tag Set (.tag) Files This file type defines the tagset of the profile, possibly by referencing other tag sets (most tag sets, for instance, will include - tagsetG and tagsetM from the Z39.50 specification. The file may + tagsetG and tagsetM from the &acro.z3950; specification. The file may contain the following directives. @@ -1323,7 +1330,7 @@ (o) The reference name of the OID for the tag set. The reference names can be found in the util - module of YAZ. + module of &yaz;. The directive is optional, since not all tag sets are registered outside of their schema. @@ -1452,9 +1459,9 @@ - +
- +
The Variant Set (.var) Files @@ -1484,7 +1491,7 @@ (o) The reference name of the OID for the variant set, if one is required. The reference names can be found - in the util module of YAZ. + in the util module of &yaz;. @@ -1533,16 +1540,16 @@ - +
- +
The Element Set (.est) Files The element set specification files describe a selection of a subset of the elements of a database record. The element selection mechanism is equivalent to the one supplied by the Espec-1 - syntax of the Z39.50 specification. + syntax of the &acro.z3950; specification. In fact, the internal representation of an element set specification is identical to the Espec-1 structure, and we'll refer you to the description of that structure for most of @@ -1556,7 +1563,7 @@ otherwise is noted. - + The directives available in the element set file are as follows: @@ -1583,7 +1590,7 @@ provides a default variant request for use when the individual element requests (see below) do not contain a variant request. Variant requests consist of a blank-separated list of - variant components. A variant compont is a comma-separated, + variant components. A variant component is a comma-separated, parenthesized triple of variant class, type, and value (the two former values being represented as integers). The value can currently only be entered as a string (this will change to depend on the definition of @@ -1673,9 +1680,9 @@ - +
- +
The Schema Mapping (.map) Files @@ -1683,21 +1690,21 @@ a schema that differs from the native schema of the record. For instance, a client might only know how to process WAIS records, while the database record is represented in a more specific schema, such as - GILS. In this module, a mapping of data to one of the MARC formats is + GILS. In this module, a mapping of data to one of the &acro.marc; formats is also thought of as a schema mapping (mapping the elements of the - record into fields consistent with the given MARC specification, prior + record into fields consistent with the given &acro.marc; specification, prior to actually converting the data to the ISO2709). This use of the - object identifier for USMARC as a schema identifier represents an + object identifier for &acro.usmarc; as a schema identifier represents an overloading of the OID which might not be entirely proper. However, it represents the dual role of schema and record syntax which - is assumed by the MARC family in Z39.50. + is assumed by the &acro.marc; family in &acro.z3950;. @@ -1723,7 +1730,7 @@ This is used, for instance, by a server receiving a request to present a record in a different schema from the native one. The name, again, is found in the oid - module of YAZ. + module of &yaz;. @@ -1737,10 +1744,10 @@ - +
- - The MARC (ISO2709) Representation (.mar) Files +
+ The &acro.marc; (ISO2709) Representation (.mar) Files This file provides rules for representing a record in the ISO2709 @@ -1749,259 +1756,16 @@ - - - - Field Structure and Character Sets - - - - In order to provide a flexible approach to national character set - handling, Zebra allows the administrator to configure the set up the - system to handle any 8-bit character set — including sets that - require multi-octet diacritics or other multi-octet characters. The - definition of a character set includes a specification of the - permissible values, their sort order (this affects the display in the - SCAN function), and relationships between upper- and lowercase - characters. Finally, the definition includes the specification of - space characters for the set. - - - - The operator can define different character sets for different fields, - typical examples being standard text fields, numerical fields, and - special-purpose fields such as WWW-style linkages (URx). - - - - The default.idx file - - The field types, and hence character sets, are associated with data - elements by the .abs files (see above). - The file default.idx - provides the association between field type codes (as used in the .abs - files) and the character map files (with the .chr suffix). The format - of the .idx file is as follows - - - - - - - index field type code - - - This directive introduces a new search index code. - The argument is a one-character code to be used in the - .abs files to select this particular index type. An index, roughly, - corresponds to a particular structure attribute during search. Refer - to . - - - - sort field code type - - - This directive introduces a - sort index. The argument is a one-character code to be used in the - .abs fie to select this particular index type. The corresponding - use attribute must be used in the sort request to refer to this - particular sort index. The corresponding character map (see below) - is used in the sort process. - - - - completeness boolean - - - This directive enables or disables complete field indexing. - The value of the boolean should be 0 - (disable) or 1. If completeness is enabled, the index entry will - contain the complete contents of the field (up to a limit), with words - (non-space characters) separated by single space characters - (normalized to " " on display). When completeness is - disabled, each word is indexed as a separate entry. Complete subfield - indexing is most useful for fields which are typically browsed (eg. - titles, authors, or subjects), or instances where a match on a - complete subfield is essential (eg. exact title searching). For fields - where completeness is disabled, the search engine will interpret a - search containing space characters as a word proximity search. - - - - charmap filename - - - This is the filename of the character - map to be used for this index for field type. - - - - - +
+
- - The character map file format - - The contents of the character map files are structured as follows: - - - - - - - lowercase value-set - - - This directive introduces the basic value set of the field type. - The format is an ordered list (without spaces) of the - characters which may occur in "words" of the given type. - The order of the entries in the list determines the - sort order of the index. In addition to single characters, the - following combinations are legal: - - - - - - - - Backslashes may be used to introduce three-digit octal, or - two-digit hex representations of single characters - (preceded by x). - In addition, the combinations - \\, \\r, \\n, \\t, \\s (space — remember that real - space-characters may not occur in the value definition), and - \\ are recognized, with their usual interpretation. - - - - - - Curly braces {} may be used to enclose ranges of single - characters (possibly using the escape convention described in the - preceding point), eg. {a-z} to introduce the - standard range of ASCII characters. - Note that the interpretation of such a range depends on - the concrete representation in your local, physical character set. - - - - - - paranthesises () may be used to enclose multi-byte characters - - eg. diacritics or special national combinations (eg. Spanish - "ll"). When found in the input stream (or a search term), - these characters are viewed and sorted as a single character, with a - sorting value depending on the position of the group in the value - statement. - - - - - - - - - uppercase value-set - - - This directive introduces the - upper-case equivalencis to the value set (if any). The number and - order of the entries in the list should be the same as in the - lowercase directive. - - - - space value-set - - - This directive introduces the character - which separate words in the input stream. Depending on the - completeness mode of the field in question, these characters either - terminate an index entry, or delimit individual "words" in - the input stream. The order of the elements is not significant — - otherwise the representation is the same as for the - uppercase and lowercase - directives. - - - - map value-set - target - - - This directive introduces a mapping between each of the - members of the value-set on the left to the character on the - right. The character on the right must occur in the value - set (the lowercase directive) of the - character set, but it may be a paranthesis-enclosed - multi-octet character. This directive may be used to map - diacritics to their base characters, or to map HTML-style - character-representations to their natural form, etc. The - map directive can also be used to ignore leading articles in - searching and/or sorting, and to perform other special - transformations. See section . - - - - - - - Ignoring leading articles - - In addition to specifying sort orders, space (blank) handling, - and upper/lowercase folding, you can also use the character map - files to make Zebra ignore leading articles in sorting records, - or when doing complete field searching. - - - This is done using the map directive in the - character map file. In a nutshell, what you do is map certain - sequences of characters, when they occur in the - beginning of a field, to a space. Assuming that the - character "@" is defined as a space character in your file, you - can do: - - map (^The\s) @ - map (^the\s) @ - - The effect of these directives is to map either 'the' or 'The', - followed by a space character, to a space. The hat ^ character - denotes beginning-of-field only when complete-subfield indexing - or sort indexing is taking place; otherwise, it is treated just - as any other character. - - - Because the default.idx file can be used to - associate different character maps with different indexing types - -- and you can create additional indexing types, should the need - arise -- it is possible to specify that leading articles should - be ignored either in sorting, in complete-field searching, or - both. - - - If you ignore certain prefixes in sorting, then these will be - eliminated from the index, and sorting will take place as if - they weren't there. However, if you set the system up to ignore - certain prefixes in searching, then these - are deleted both from the indexes and from query terms, when the - client specifies complete-field searching. This has the effect - that a search for 'the science journal' and 'science journal' - would both produce the same results. - - - - - - - GRS Exchange Formats +
+ &acro.grs1; Exchange Formats Converting records from the internal structure to an exchange format @@ -2013,7 +1777,7 @@ - GRS-1. The internal representation is based on GRS-1/XML, so the + &acro.grs1;. The internal representation is based on &acro.grs1;/&acro.xml;, so the conversion here is straightforward. The system will create applied variant and supported variant lists as required, if a record contains variant information. @@ -2022,34 +1786,34 @@ - XML. The internal representation is based on GRS-1/XML so - the mapping is trivial. Note that XML schemas, preprocessing + &acro.xml;. The internal representation is based on &acro.grs1;/&acro.xml; so + the mapping is trivial. Note that &acro.xml; schemas, preprocessing instructions and comments are not part of the internal representation - and therefore will never be part of a generated XML record. - Future versions of the Zebra will support that. + and therefore will never be part of a generated &acro.xml; record. + Future versions of the &zebra; will support that. - SUTRS. Again, the mapping is fairly straightforward. Indentation + &acro.sutrs;. Again, the mapping is fairly straightforward. Indentation is used to show the hierarchical structure of the record. All - "GRS" type records support both the GRS-1 and SUTRS + "&acro.grs1;" type records support both the &acro.grs1; and &acro.sutrs; representations. - + - ISO2709-based formats (USMARC, etc.). Only records with a + ISO2709-based formats (&acro.usmarc;, etc.). Only records with a two-level structure (corresponding to fields and subfields) can be directly mapped to ISO2709. For records with a different structuring - (eg., GILS), the representation in a structure like USMARC involves a + (e.g., GILS), the representation in a structure like &acro.usmarc; involves a schema-mapping (see ), to an - "implied" USMARC schema (implied, + "implied" &acro.usmarc; schema (implied, because there is no formal schema which specifies the use of the - USMARC fields outside of ISO2709). The resultant, two-level record is + &acro.usmarc; fields outside of ISO2709). The resultant, two-level record is then mapped directly from the internal representation to ISO2709. See the GILS schema definition files for a detailed example of this approach. @@ -2072,7 +1836,7 @@ - + SOIF. Support for this syntax is experimental, and is currently @@ -2082,15 +1846,341 @@ level. - + - +
- - +
+ Extended indexing of &acro.marc; records + + Extended indexing of &acro.marc; records will help you if you need index a + combination of subfields, or index only a part of the whole field, + or use during indexing process embedded fields of &acro.marc; record. + + + Extended indexing of &acro.marc; records additionally allows: + + + + to index data in LEADER of &acro.marc; record + + + + to index data in control fields (with fixed length) + + + + to use during indexing the values of indicators + + + + to index linked fields for UNI&acro.marc; based formats + + + + + + In compare with simple indexing process the extended indexing + may increase (about 2-3 times) the time of indexing process for &acro.marc; + records. + +
+ The index-formula + + At the beginning, we have to define the term + index-formula for &acro.marc; records. This term helps + to understand the notation of extended indexing of &acro.marc; records by &zebra;. + Our definition is based on the document + "The table + of conformity for &acro.z3950; use attributes and R&acro.usmarc; fields". + The document is available only in Russian language. + + + The index-formula is the combination of + subfields presented in such way: + + + + 71-00$a, $g, $h ($c){.$b ($c)} , (1) + + + + We know that &zebra; supports a &acro.bib1; attribute - right truncation. + In this case, the index-formula (1) consists from + forms, defined in the same way as (1) + + + 71-00$a, $g, $h + 71-00$a, $g + 71-00$a + + + + The original &acro.marc; record may be without some elements, which included in index-formula. + + + + This notation includes such operands as: + + + + # + It means whitespace character. + + + + - + The position may contain any value, defined by + &acro.marc; format. + For example, index-formula + + + 70-#1$a, $g , (2) + + + includes + + + 700#1$a, $g + 701#1$a, $g + 702#1$a, $g + + + + + + + {...} + + The repeatable elements are defined in figure-brackets {}. + For example, + index-formula + + + 71-00$a, $g, $h ($c){.$b ($c)} , (3) + + + includes + + + 71-00$a, $g, $h ($c). $b ($c) + 71-00$a, $g, $h ($c). $b ($c). $b ($c) + 71-00$a, $g, $h ($c). $b ($c). $b ($c). $b ($c) + + + + + + + + + All another operands are the same as accepted in &acro.marc; world. + + + +
+ +
+ Notation of <emphasis>index-formula</emphasis> for &zebra; + + + Extended indexing overloads path of + elm definition in abstract syntax file of &zebra; + (.abs file). It means that names beginning with + "mc-" are interpreted by &zebra; as + index-formula. The database index is created and + linked with access point (&acro.bib1; use attribute) + according to this formula. + + For example, index-formula + + + 71-00$a, $g, $h ($c){.$b ($c)} , (4) + + + in .abs file looks like: + + + mc-71.00_$a,_$g,_$h_(_$c_){.$b_(_$c_)} + + + + The notation of index-formula uses the operands: + + + + _ + It means whitespace character. + + + + . + The position may contain any value, defined by + &acro.marc; format. For example, + index-formula + + + 70-#1$a, $g , (5) + + + matches mc-70._1_$a,_$g_ and includes + + + 700_1_$a,_$g_ + 701_1_$a,_$g_ + 702_1_$a,_$g_ + + + + + + {...} + The repeatable elements are defined in + figure-brackets {}. For example, + index-formula + + + 71#00$a, $g, $h ($c) {.$b ($c)} , (6) + + + matches + mc-71.00_$a,_$g,_$h_(_$c_){.$b_(_$c_)} and + includes + + + 71.00_$a,_$g,_$h_(_$c_).$b_(_$c_) + 71.00_$a,_$g,_$h_(_$c_).$b_(_$c_).$b_(_$c_) + 71.00_$a,_$g,_$h_(_$c_).$b_(_$c_).$b_(_$c_).$b_(_$c_) + + + + + + <...> + Embedded index-formula (for + linked fields) is between <>. For example, + index-formula + + + + 4--#-$170-#1$a, $g ($c) , (7) + + + matches + mc-4.._._$1<70._1_$a,_$g_(_$c_)>_ and + includes + + + 463_._$1<70._1_$a,_$g_(_$c_)>_ + + + + + + + + + All another operands are the same as accepted in &acro.marc; world. + + +
+ Examples + + + + + + indexing LEADER + + You need to use keyword "ldr" to index leader. For example, + indexing data from 6th and 7th position of LEADER + + + elm mc-ldr[6] Record-type ! + elm mc-ldr[7] Bib-level ! + + + + + + + indexing data from control fields + + indexing date (the time added to database) + + + elm mc-008[0-5] Date/time-added-to-db ! + + + or for R&acro.usmarc; (this data included in 100th field) + + + elm mc-100___$a[0-7]_ Date/time-added-to-db ! + + + + + + + using indicators while indexing + + For R&acro.usmarc; index-formula + 70-#1$a, $g matches + + + elm 70._1_$a,_$g_ Author !:w,!:p + + + When &zebra; finds a field according to + "70." pattern it checks the indicators. In this + case the value of first indicator doesn't mater, but the value of + second one must be whitespace, in another case a field is not + indexed. + + + + + indexing embedded (linked) fields for UNI&acro.marc; based + formats + + For R&acro.usmarc; index-formula + 4--#-$170-#1$a, $g ($c) matches + + _ Author !:w,!:p + ]]> + + Data are extracted from record if the field matches to + "4.._." pattern and data in linked field + match to embedded + index-formula + 70._1_$a,_$g_(_$c_). + + + + + + + +
+
+ +
+ + +