1 Update code to use new YAZ log functions/defines.
5 Fixed several compilation warnings. (gcc 4.1.2, -O3 -g -Wall)
7 Fixed bug #710: Duplicate keys for CDATA in xelm/melm rules.
11 Fixed bug #529: multiple simultaneous updates by extended services trashes
16 Fixed bug #47: Commit needs to check for roll-back.
18 Fixed bug #672: Trailing characters in password are ignored
20 Added extra presence check for tcl.h, because some systems have
21 tclConfig.sh installed even though Tcl C headers are missing.
25 Optimized melm performance.
27 Do not use sync(2) during commit (but rely on sync'd individual files).
31 Updated Debian package to use sym links for zebraidx, zebrasrv.
33 Fixed bug in isamb (isam:b) regarding ISAM tree splitting.
35 Fixed problem with file locking during commit phase. This error was
40 Fixed bug #602: Any extra terms appended to a wordlist with a hitcount
41 of 0, gets incorrect term count
43 Fixed bug #597: Support null missing key for sort.
47 Fixed bug #465: dup fields in ISO2709 retrieval.
49 Another fix for X-Path indexing. The previous fix, unfortunately, indexed
54 Fix X-Path attribute indexing. Bug #431.
58 Fixed bug #415: Strange truncation behavior.
60 Added 'melm' directive to absyn format to simplify config files
61 for MARC-style databases. See tab/marc21.abs for an example.
63 Added support for special slement set _sysno_ which returns a
64 record ID for a record packed as a SUTRS record.
68 Documented authentication facility in Zebra. Added zebra.cfg directive
69 'passwd.c' which specifies user accounts file with encrypted passwords. The
70 directive 'passwd' specifies user accounts file with clear-text passwords.
71 The previous version of Zebra used plain/clear text depending on
72 configuration automatically. That caused upgrade trouble. Bug #356.
76 Depend on YAZ 2.0.18 or later in configure.
78 Fixed crash that could occur if ES update transaction failed.
80 Configure enables the use of the crypt API - if available.
82 Fixed bug #304: Fuzzy search regExpr-2 did not use proper error distance
85 Fixed bug #305: Scan now handles negative preferred-position-in-response.
87 Implemented the 'equivalent' directive for .chr-files.
91 For configure, support threading again. It was removed by mistake
94 Fixed bug #262: spaces in control fields in MARC returned.
96 Fixed bug #259: Second indicator lost in MARC records
100 Fixed bug #253: Setting group.database not honored
102 Fixed bug #252: Sort does not work.
104 Fixed bug #248: hit counts in combinatoric (and) searches in specific ..
106 --- 1.3.20 2005/01/17
108 Fixed bug #245: Time for getting records changes a lot based on record
111 Fixed bug #235: weird x-path results.
113 Avoid crash in ISAMB when isamb_pp_open gets ISAMB_POS = 0. In
114 this case EOF (no entries) is signalled. It fixes problem with
115 terms being deleted. See bug #109.
117 Fixed bug #169: Phrase term counts does not work. The bug exists in
120 Added mechanism to ignore leading articles when doing full-field indexing,
121 based on the character map files. See the manual for further discussion.
123 --- 1.3.18 2004/08/20
125 Fixed bug in record management. Releasing blocks could result in
128 Fixed bug in isam:b. A tree split could result in a lost item.
130 --- 1.3.17 2004/08/17
132 Add IDZebra.i to dist so that Perl extension builds again.
134 --- 1.3.16 2004/08/16
136 Added facility to make attibutes in grs.regx and grs.tcl filter using the
137 data command with argument -attribute <name> . The content of data is
138 the value of the attribute. This command should be used inside a
139 begin element , end element section.
141 Update zebra.nsi to NSIS 2.
143 Added a new 'cut' directive to charmaps (.chr files) which specifies that
144 only characters after the cutting char should be indexed.
146 Update Perl internals so that it matches the current Zebra API.
147 The recordGroup structure is no longer available. A group of resources
148 can still be referenced by setting groupName=>.. in various methods.
150 Maximum number of records to be sorted in a result set can be
151 specified by setting "sortmax". Default is 1000.
153 Allow use of string use attributes for regular attribute sets. The
154 name matches the name given in the attribute set file. All strings
155 starting with / are considered X-Path as usual.
157 Fixed bug in grs.regx. filter . 'end element' could pop off top tag
158 element for XML tree. It may only pop off if -record is given.
160 Added grs.danbib filter - for Danish Bibliographic Centre.
162 Rename CHANGELOG to NEWS.
164 For text filter, return only header if elementSetName=H . elementSetName=R
165 returns contents only. Other elementSetName returns both header+content.
167 Added test for charmap and rusmarc.
169 Added feature charmaps (.chr) so that characters may be specified in
172 Fixed problem with encoding directive for charmap(.chr) files.
174 Allow Remote insert/delete/replace/update with record, recordIdNumber
175 (sysno) and/or recordIdOpaque(user supplied record Id). If both
176 IDs are omitted internal record ID match is assumed (recordId: - in
179 --- 1.3.15 2004/01/15
181 Fix bug. X-Path attribute expressions with spaces in them now works.
183 Fix base address for MARC output.
185 --- 1.3.14 2003/11/29
187 Fix bug with shadow and result set handling.
189 Implement MARCXML to ISO2709 conversion.
191 --- 1.3.13 2003/09/26
193 Add missing examples for Windows install.
195 Fix bug in regx filter to make it "greedy" again. This bug appeared
200 --- 1.3.12 2003/09/08
202 Fix XML error handling. Stop XML parse immediately if XML parse error
203 occur (i.e. produce one error only).
205 Zebra ignores "unsupported use attribute" for individual databases
206 when search multiple databases (unless all databases fail).
208 New filter grs.marcxml which works like grs.marc but produces MARCXML.
210 Added support for database deletion. It is possible to create/drop
211 a database from zebraidx utility. Note: only for isam:b.
213 Write zebrasrv.pid to lockdir.
215 Bug fix: result sets were not recovered correctly. Had to
216 add ODR handle for zebra_search_RPN in order to make it work.
218 Fixed a bug in regx filters that didn't do anchors (^) correctly.
220 Fixed a bug in searches with X-Path searches sometimes giving "extra"
223 Zebra server checks for zebrasrv.pid and refuses to start if it is already
224 locked by another (running) zebrasrv.
226 Fixed a bug with text being chunked in pieces for the grs.xml filter.
228 --- 1.3.11 2003/04/25
230 xelm code updates. xelm works regardless state of 'xpath enable/disable'
231 Avoid -L/usr/lib since that is already default library path.
233 Allow multiple updates within one transaction.
235 Fixed a bug with >2GB files (overflow in integer expression).
237 --- 1.3.10 2003/04/01
239 Fix linker error for Perl module.
241 Fix bug in and operation which in some cases could result in "extra"
242 hits. Bug was introduced in 1.3.5.
244 Fix bug in handling of schema conversion when producing numeric tags.
250 Add missing files doc/zvrank.txt and doc/marc_indexing.xml.
254 Zvrank: an experimental ranking algorithm. See doc/zvrank.txt and
255 source in index/zvrank.c. Enable this by using rank: zvrank in zebra.cfg.
256 Contributed by Johannes Leveling <Johannes.Leveling at fernuni-hagen.de>
258 livrank: another experimental ranking algorithm. Source in livcode.c.
259 Enable this by using rank: livrank in zebra.cfg and use -DLIV_CODE=1
261 Contributed by Pete Mallinson, University of Liverpool.
263 Advanced MARC indexing. See doc/marc_indexing.xml
264 Oleg Kolobov <oleg at lib.tpu.ru>
266 Perl API updates and fixes.
267 Peter Popovics <pop at technomat.hu>
269 Fixed 'zebraidx delete'.
271 Implemented 'zebraidx clean'.
273 64-bit offsets for register files on WIN32 (no 2 GB limit).
275 Fixed a few memory leaks WRT sorting.
279 Fixed error handling : error code was not properly returned.
281 Support Truncation 104 (CCL).
285 Added missing source files for perl extension.
289 Implemented xelm directive.
291 Updated for newer version of YAZ (introduction of string schema).
293 Directory examples/zthes now part of distribution (was missing
294 in previous release).
296 New .abs directive, systag, that control where to put retrieval
297 information. The directive takes two arguments: system tag, element name.
298 System tag is one of : rank, sysno, size.
302 Perl Filter and Perl API. By Peter Popovics.
304 For zebra.cfg, if no profilePath is specified, directory
305 (prefix)/share/idzebra/tab
308 Zebra Examples in examples . Zebra tests in test.
310 Bug fix: sort index was not properly modified on
311 record updates/deletes.
313 Fix handling of character entities for sgml filter.
315 Move data1 to Zebra (used to be part of YAZ).
319 Fix character encoding of scan response terms.
321 Fix character decoding of scan request terms.
323 Fix ESpec handling (requires YAZ 1.9.1)
325 Fix searches for complete fields.
329 When name zebra is used in a filename or directory 'idzebra' is used
330 instead to avoid confusion with GNU zebra (routing software).
332 Zebra server stops with a fatal error if config file cannot be read.
334 New config setting, followLinks, that controls whether update of files
335 should follow symbolic. Set it to 1 (for enable) or 0 (to disable).
336 By default symbolic links are followed.
338 Fix MARC transfer . MARC fields had wrong data for multiple fields.
340 XML record reader moved from YAZ to Zebra, to make YAZ less
341 dependant on external libraries.
343 Zebra uses yaz_iconv which is mini iconv library supporting UTF-8,
344 UCS4, ISO-8859-1. This means that Zebra does UNICODE even
345 on systems that doesn't offer iconv.
347 XML record reader supports external system entities.
351 New .abs-directive "xpath" that takes one argument: "enable"
352 or "disable" to enable and disable XPath -indexing. If no "xpath"
353 direcive is found in .abs-file , XPath-indexing is disabled to ensure
354 backwards compatibility. For missing .abs-files XPath-indexing is
355 enabled so that such records are searchable.
357 Zebra warns about missing .abs-file only once (for each type).
359 Fixed a bug in file update where already-inserted files could
364 Zebra license changed to GNU GPL.
366 XPath-like queries used when RPN string attributes are used, eg.
367 @attr 1=/portal/title sometitle
368 @attr 1=/portal/title[@xml:lang=da] danishtitle
369 @attr 1=/portal/title/@xml:lang da
370 @attr 1=//title sometitle
372 Zebra uses UTF-8 internally:
373 1) New setting "encoding" for zebra.cfg that specifies encoding for
374 OCTET terms in queries and record encoding for most transfer syntaxes
375 (except those that use International Strings, such as GRS-1).
376 2) The encoding of International strings is UTF-8 by default. It
377 may be changed by character set negotiation. If character set
378 negotiation is in effect and if records are selected for conversion
379 these'll be converted to the selected character set - thus overriding
380 the encoding setting in zebra.cfg.
381 3) New directive "encoding" in .abs-files. This specifies the external
382 character encoding for files indexed by zebra. However, if records
383 themselves have an XML header that specifies and encoding that'll be used
386 XML filter (-t grs.xml).
388 Multiple registers. New setting in resource 'root' that holds base
389 directory for register(s). A group a databases may be put in separate
390 register in directory root/reg by using db name 'reg/db1' ... 'reg/dbN'.
394 Fixes for Digital Unix
396 Implemented hits per term using USR:SearchResult-1.
398 New Zebra API. Locking system re-implemented.
400 --- 1.1.stable 2002/02/20
402 Rank weight can be controlled with attribute type 9. Default
403 value is 34. Recommended values between 1-36.
407 Updated for YAZ version 1.8.
409 Added support for termsets - a result set of terms matching
410 a given query. For @attr 8=<set> creates termset named <set>.
412 Added support for raw retrieval. Element Set Name R forces the
413 text filter which returns the record in its original form.
415 Added numerical sort - triggered by structure=numeric (4=109).
417 Remote record import using Z39.50 Extended Services and Segments.
419 Fixed bug where updating a database with user-defined attributes
420 could corrupt the register (bad storeKeys).
422 Multi-threaded version.
424 Fixed bug regarding proximity.
426 Documentation updates.
428 Fixed bug in record retrieval module that occured on 64-bit OSF
433 Fixed bug in makefile for WIN32.
435 Fixed bug in configure script - used bash-specific features.
439 Added support for multiple records in one file for filter grs.sgml.
441 Changed record index structure. New layout is incompatible with
442 previous releases. Added setting "recordcompression" to control
443 compression of records. Possible values are "none" (no
444 compression) and bzip2 (compression using libbz2).
446 Added XML transfer syntax support for retrieval of structured records.
447 Schema in CompSpec is recognised in retrieval of structured records.
449 Changed Tcl record filter so that it attemps to read <filt>.tflt. If
450 that fails, the filter reads the file <filt>.flt (regx style filter).
452 Implemented new Tcl record filter - use grs.tcl.<filter> to enable it.
453 Zebra's configure script automatically attempts to locate Tcl. For
454 manual Tcl configuration use option --with-tclconfig=<path> to specify
455 where Tcl's library files are located.
457 Implemented "compression" of Dictionary and ISAM system. Dictionary
460 Added "tagsysno" directive to zebra.cfg to control under which tag the
461 system ID is placed. Use tagsysno: 0 to disable Zebra's system number
464 Added "tagrank" as above.
466 Changed file naming scheme for register files from <name>.mf.<no> to
469 Implemented "position"-flag for register type (as defined in
470 default.idx). When set to zero no position (or seqence number) is
471 saved in register for each word occurrence, thus saving some register
474 Implemented database mapping. Using mapdb one can specify a database
475 to be mapped to one or more physical databases. Usage:
476 mapdb <fromdb> <todb> ..
478 Added SOIF-filter. Thanks to Peter Valkenburg.
480 For the regx-filter "end element -record" may trigger a mark-of-record
481 if outer level is reached.
483 Tag sets may be typed in the reference to it. From the .abs-file the
484 "tagset" directive takes a third optional integer type for the tag set
485 referenced. From a .tag-file the "include" directive takes a third
486 optional type as well. The old "type" directive in the tag set itself
487 is still recognized but acts as the default type for the tag set.
489 Zebra supports the specification of arbitrary attributes sets, schemas
490 and tag sets, because of the change in YAZ' OID management system.
492 Fixed bug in Sort that caused it NOT to use character mapping as it
495 Zebra now uses GNU configure to generate Makefile(s).
497 Added un-optimised support for left and left/right truncation attributes.
499 Added support for relational operators on text when using RPN queries.
501 Added support for sort specifications in RPN queries. Type 7 specifies
502 'sort' where value 1=ascending, value 2=descending. The use attribute
503 specifies the field criteria as usual. The term specifies priority
504 where 0=first, 1=second, ...
506 Changed the way use attributes are specified in the recordId
509 Maximum number of databases in one Zebra register increased.
511 New setting, databasePath, which specifies that first directory during
512 update traversal is the database name (instead of a fixed one).
514 New setting, explainDatabase, which specifies that databases are
517 Modified Zebra so that it works with ASN.1 compiled code for YAZ.
519 Implemented EXPLAIN database maintenance. Zebra automatically
520 generate - and update CategoryList, TargetInfo, DatabaseInfo,
521 AttributeSetInfo and AttributeDetails records at this stage. The
522 records may be transferred as GRS-1, SUTRS or Explain.
524 Fixed register spec so that colon isn't treated as size separator
525 unless followed by [0-9+-] in order to allow DOS drive specifications.
527 Fixed two bugs in ISAMC system.
529 Changed the way Zebra keeps its maintenance information about attribute
530 sets, available attributes, etc.. Records in "SGML" notation using an
531 EXPLAIN schema is now used when appropriate.
533 Bug fix: Index didn't handle update/insert/delete of the same record
534 (i.e. same recordId) in one run (one invocation of zebraidx). Only the
535 first occurence of a record is considered.
537 Most searches now return correct number of hits.
539 New modular ranking system. Interested programmers are encouraged to
540 inspect rank1.c and improve the algorithm.
542 Bug fix: Lock files weren't removed as they should on NT.
544 Implemented Z39.50 Sort. Zebra's sort handler uses use attributes to
545 specify a "sort register". Refer to the gils sample records which refer
546 to index type "s" which is specified as "sort" in the default.idx file.
547 Each sort criteria can either be Ascending or Descending and at most
548 three sort elements can be specified.
550 Bug fix: Character mapping didn't work for text files.
554 Simple ranked searches now return correct number of hits.
556 The test option (-s) only makes a read-lock on the index as well
557 as using read-only operations anywhere.
559 Moved towards generic character mapping. Configuration file default.idx
560 specifies character map files for register types w, p, u, etc.
562 Implemented "begin variant" for the sgml.regx - filter.
564 Fixed a few memory leaks.
566 Added support for C++, headers uses extern "C" for public definitions.
568 Bug fix: The show records facility (-s) only displayed information for
569 the first record in a file (and not for every record in the file).
571 Added option "-f <n>" to limit the logging of record operations. After
572 <n> records has been processed no logging is performed (unless errors
575 Bug fix: the compressed ISAM system didn't handle update operations
578 Added setting, "maxResultSetSize", to hold the number of records to
579 save in a result set.
581 Bug fix: Complete phrase did't work for search operations.
583 Bug fix: temporary result sets weren't deleted.
585 Reduced disk space for saved keys (storeKeys = 1).
587 Added optional, physical ANY (key replication)
589 Implemented proximity operator in search.
591 Bug fix: the path name buffers used by file match traversal routines
592 have been extended to support long file names.
594 New C(ompressed) ISAM system. To enable it, specify "isam: c" in the
595 configuration file. The resulting register without "storeKeys" is about
596 half the size, and the memory used by zebraidx during phase 2 (merge) is
597 reduced to a minimum.
599 Reworked the way Regexp-2 queries with error tolerance are handled and
600 specified. The documentation has been updated accordingly.
602 Bug fix: Zebrasrv didn't search correctly when queries contained masking
603 characters. This bug was introduced in 1.0a8.
605 Zebrasrv now tag records with the proper database name.
607 New settings, memMax and keyTmpDir.
609 Changed name of setting lockDir (previously called lockPath) and
610 setTmpDir (previously called tempSetPath).
612 Generalized and changed record type specifications. In short, there are:
614 grs.sgml structured, "SGML-like" syntax
615 grs.regx.<filter> structured, Regular expression filter
616 grs.marc.<abs> Reads *MARC records in the ISO2709 format. <abs>
617 is the name of an abstract syntax file.
618 Bug fix: Result sets weren't sorted in operations involving boolean
619 operations with "ranked" operands.
623 Added national character-handling subsystem.
627 Small modifications to input filters and profiles.
629 Added support for SOIF syntax (with private OID).
633 Fixed buffer-size problem in indexing.
635 Added compression to temporary files for updating.
637 Added phrase registers.
639 Added dynamic mapping of search attribute to multiple termlists (ANY).
641 Scan support in multiple databases/registers.
643 Configuration settings are case-insensitive and single dash (-)
644 characters are ignored in comparisons.
646 The index processing ignores empty files - warning given.
648 New option to zebraidx (-V) displays version information.
652 Fixed problem in file-update system.
654 Fixed problem in shadow system; register was sometimes corrupted after
659 Fixed problems in the ISAM subsystem. Caused difficulties when updating
662 Fixed small problem in SUTRS-filter. A newline was sometimes inserted before
663 the rank and record number.
665 Fixed bug in the isam subsystem - caused a malfunction when accessing
666 words which occurred more than 10000 times.
668 Distribution should now include YAZ (Z39.50 protocol stack) to simplify
671 Server can now run under inetd. Use option -i, and -w <directory> to
672 set working directory to desired location.
674 New zebraidx command: clean - removes temporary shadow files.
676 Fixed bug in ISAM system. Occurred rarely during register updates.
678 Logging during index merge phase is improved. The remaining running
681 Temporary files generated by zebraidx are removed after each run.
683 Bug fix: Dictionary didn't handle 8-bit characters correctly; was obvious
684 when doing scan operations in dictionaries with European characters.
688 A whole slew of updates, to make the first publicized release. Get the doc
693 Memory-problems in ISAM fixed. More blocktypes added to the default setup
694 to increase performance on larger databases.
696 Various minor changes in data management system.
700 A couple of portability-problems resolved.
702 Changed some malloc() to xmalloc().