3 Fixed buffer overrun in leaf page merge code in ISAMB system.
5 Update code to use new YAZ log functions/defines.
9 Updated to use YLOG_-defines for YAZ, rather than the obsolte LOG-defines.
13 Fixed several compilation warnings. (gcc 4.1.2, -O3 -g -Wall)
15 Fixed bug #710: Duplicate keys for CDATA in xelm/melm rules.
19 Fixed bug #529: multiple simultaneous updates by extended services trashes
24 Fixed bug #47: Commit needs to check for roll-back.
26 Fixed bug #672: Trailing characters in password are ignored
28 Added extra presence check for tcl.h, because some systems have
29 tclConfig.sh installed even though Tcl C headers are missing.
33 Optimized melm performance.
35 Do not use sync(2) during commit (but rely on sync'd individual files).
39 Updated Debian package to use sym links for zebraidx, zebrasrv.
41 Fixed bug in isamb (isam:b) regarding ISAM tree splitting.
43 Fixed problem with file locking during commit phase. This error was
48 Fixed bug #602: Any extra terms appended to a wordlist with a hitcount
49 of 0, gets incorrect term count
51 Fixed bug #597: Support null missing key for sort.
55 Fixed bug #465: dup fields in ISO2709 retrieval.
57 Another fix for X-Path indexing. The previous fix, unfortunately, indexed
62 Fix X-Path attribute indexing. Bug #431.
66 Fixed bug #415: Strange truncation behavior.
68 Added 'melm' directive to absyn format to simplify config files
69 for MARC-style databases. See tab/marc21.abs for an example.
71 Added support for special slement set _sysno_ which returns a
72 record ID for a record packed as a SUTRS record.
76 Documented authentication facility in Zebra. Added zebra.cfg directive
77 'passwd.c' which specifies user accounts file with encrypted passwords. The
78 directive 'passwd' specifies user accounts file with clear-text passwords.
79 The previous version of Zebra used plain/clear text depending on
80 configuration automatically. That caused upgrade trouble. Bug #356.
84 Depend on YAZ 2.0.18 or later in configure.
86 Fixed crash that could occur if ES update transaction failed.
88 Configure enables the use of the crypt API - if available.
90 Fixed bug #304: Fuzzy search regExpr-2 did not use proper error distance
93 Fixed bug #305: Scan now handles negative preferred-position-in-response.
95 Implemented the 'equivalent' directive for .chr-files.
99 For configure, support threading again. It was removed by mistake
102 Fixed bug #262: spaces in control fields in MARC returned.
104 Fixed bug #259: Second indicator lost in MARC records
106 --- 1.3.22 2005/01/23
108 Fixed bug #253: Setting group.database not honored
110 Fixed bug #252: Sort does not work.
112 Fixed bug #248: hit counts in combinatoric (and) searches in specific ..
114 --- 1.3.20 2005/01/17
116 Fixed bug #245: Time for getting records changes a lot based on record
119 Fixed bug #235: weird x-path results.
121 Avoid crash in ISAMB when isamb_pp_open gets ISAMB_POS = 0. In
122 this case EOF (no entries) is signalled. It fixes problem with
123 terms being deleted. See bug #109.
125 Fixed bug #169: Phrase term counts does not work. The bug exists in
128 Added mechanism to ignore leading articles when doing full-field indexing,
129 based on the character map files. See the manual for further discussion.
131 --- 1.3.18 2004/08/20
133 Fixed bug in record management. Releasing blocks could result in
136 Fixed bug in isam:b. A tree split could result in a lost item.
138 --- 1.3.17 2004/08/17
140 Add IDZebra.i to dist so that Perl extension builds again.
142 --- 1.3.16 2004/08/16
144 Added facility to make attibutes in grs.regx and grs.tcl filter using the
145 data command with argument -attribute <name> . The content of data is
146 the value of the attribute. This command should be used inside a
147 begin element , end element section.
149 Update zebra.nsi to NSIS 2.
151 Added a new 'cut' directive to charmaps (.chr files) which specifies that
152 only characters after the cutting char should be indexed.
154 Update Perl internals so that it matches the current Zebra API.
155 The recordGroup structure is no longer available. A group of resources
156 can still be referenced by setting groupName=>.. in various methods.
158 Maximum number of records to be sorted in a result set can be
159 specified by setting "sortmax". Default is 1000.
161 Allow use of string use attributes for regular attribute sets. The
162 name matches the name given in the attribute set file. All strings
163 starting with / are considered X-Path as usual.
165 Fixed bug in grs.regx. filter . 'end element' could pop off top tag
166 element for XML tree. It may only pop off if -record is given.
168 Added grs.danbib filter - for Danish Bibliographic Centre.
170 Rename CHANGELOG to NEWS.
172 For text filter, return only header if elementSetName=H . elementSetName=R
173 returns contents only. Other elementSetName returns both header+content.
175 Added test for charmap and rusmarc.
177 Added feature charmaps (.chr) so that characters may be specified in
180 Fixed problem with encoding directive for charmap(.chr) files.
182 Allow Remote insert/delete/replace/update with record, recordIdNumber
183 (sysno) and/or recordIdOpaque(user supplied record Id). If both
184 IDs are omitted internal record ID match is assumed (recordId: - in
187 --- 1.3.15 2004/01/15
189 Fix bug. X-Path attribute expressions with spaces in them now works.
191 Fix base address for MARC output.
193 --- 1.3.14 2003/11/29
195 Fix bug with shadow and result set handling.
197 Implement MARCXML to ISO2709 conversion.
199 --- 1.3.13 2003/09/26
201 Add missing examples for Windows install.
203 Fix bug in regx filter to make it "greedy" again. This bug appeared
208 --- 1.3.12 2003/09/08
210 Fix XML error handling. Stop XML parse immediately if XML parse error
211 occur (i.e. produce one error only).
213 Zebra ignores "unsupported use attribute" for individual databases
214 when search multiple databases (unless all databases fail).
216 New filter grs.marcxml which works like grs.marc but produces MARCXML.
218 Added support for database deletion. It is possible to create/drop
219 a database from zebraidx utility. Note: only for isam:b.
221 Write zebrasrv.pid to lockdir.
223 Bug fix: result sets were not recovered correctly. Had to
224 add ODR handle for zebra_search_RPN in order to make it work.
226 Fixed a bug in regx filters that didn't do anchors (^) correctly.
228 Fixed a bug in searches with X-Path searches sometimes giving "extra"
231 Zebra server checks for zebrasrv.pid and refuses to start if it is already
232 locked by another (running) zebrasrv.
234 Fixed a bug with text being chunked in pieces for the grs.xml filter.
236 --- 1.3.11 2003/04/25
238 xelm code updates. xelm works regardless state of 'xpath enable/disable'
239 Avoid -L/usr/lib since that is already default library path.
241 Allow multiple updates within one transaction.
243 Fixed a bug with >2GB files (overflow in integer expression).
245 --- 1.3.10 2003/04/01
247 Fix linker error for Perl module.
249 Fix bug in and operation which in some cases could result in "extra"
250 hits. Bug was introduced in 1.3.5.
252 Fix bug in handling of schema conversion when producing numeric tags.
258 Add missing files doc/zvrank.txt and doc/marc_indexing.xml.
262 Zvrank: an experimental ranking algorithm. See doc/zvrank.txt and
263 source in index/zvrank.c. Enable this by using rank: zvrank in zebra.cfg.
264 Contributed by Johannes Leveling <Johannes.Leveling at fernuni-hagen.de>
266 livrank: another experimental ranking algorithm. Source in livcode.c.
267 Enable this by using rank: livrank in zebra.cfg and use -DLIV_CODE=1
269 Contributed by Pete Mallinson, University of Liverpool.
271 Advanced MARC indexing. See doc/marc_indexing.xml
272 Oleg Kolobov <oleg at lib.tpu.ru>
274 Perl API updates and fixes.
275 Peter Popovics <pop at technomat.hu>
277 Fixed 'zebraidx delete'.
279 Implemented 'zebraidx clean'.
281 64-bit offsets for register files on WIN32 (no 2 GB limit).
283 Fixed a few memory leaks WRT sorting.
287 Fixed error handling : error code was not properly returned.
289 Support Truncation 104 (CCL).
293 Added missing source files for perl extension.
297 Implemented xelm directive.
299 Updated for newer version of YAZ (introduction of string schema).
301 Directory examples/zthes now part of distribution (was missing
302 in previous release).
304 New .abs directive, systag, that control where to put retrieval
305 information. The directive takes two arguments: system tag, element name.
306 System tag is one of : rank, sysno, size.
310 Perl Filter and Perl API. By Peter Popovics.
312 For zebra.cfg, if no profilePath is specified, directory
313 (prefix)/share/idzebra/tab
316 Zebra Examples in examples . Zebra tests in test.
318 Bug fix: sort index was not properly modified on
319 record updates/deletes.
321 Fix handling of character entities for sgml filter.
323 Move data1 to Zebra (used to be part of YAZ).
327 Fix character encoding of scan response terms.
329 Fix character decoding of scan request terms.
331 Fix ESpec handling (requires YAZ 1.9.1)
333 Fix searches for complete fields.
337 When name zebra is used in a filename or directory 'idzebra' is used
338 instead to avoid confusion with GNU zebra (routing software).
340 Zebra server stops with a fatal error if config file cannot be read.
342 New config setting, followLinks, that controls whether update of files
343 should follow symbolic. Set it to 1 (for enable) or 0 (to disable).
344 By default symbolic links are followed.
346 Fix MARC transfer . MARC fields had wrong data for multiple fields.
348 XML record reader moved from YAZ to Zebra, to make YAZ less
349 dependant on external libraries.
351 Zebra uses yaz_iconv which is mini iconv library supporting UTF-8,
352 UCS4, ISO-8859-1. This means that Zebra does UNICODE even
353 on systems that doesn't offer iconv.
355 XML record reader supports external system entities.
359 New .abs-directive "xpath" that takes one argument: "enable"
360 or "disable" to enable and disable XPath -indexing. If no "xpath"
361 direcive is found in .abs-file , XPath-indexing is disabled to ensure
362 backwards compatibility. For missing .abs-files XPath-indexing is
363 enabled so that such records are searchable.
365 Zebra warns about missing .abs-file only once (for each type).
367 Fixed a bug in file update where already-inserted files could
372 Zebra license changed to GNU GPL.
374 XPath-like queries used when RPN string attributes are used, eg.
375 @attr 1=/portal/title sometitle
376 @attr 1=/portal/title[@xml:lang=da] danishtitle
377 @attr 1=/portal/title/@xml:lang da
378 @attr 1=//title sometitle
380 Zebra uses UTF-8 internally:
381 1) New setting "encoding" for zebra.cfg that specifies encoding for
382 OCTET terms in queries and record encoding for most transfer syntaxes
383 (except those that use International Strings, such as GRS-1).
384 2) The encoding of International strings is UTF-8 by default. It
385 may be changed by character set negotiation. If character set
386 negotiation is in effect and if records are selected for conversion
387 these'll be converted to the selected character set - thus overriding
388 the encoding setting in zebra.cfg.
389 3) New directive "encoding" in .abs-files. This specifies the external
390 character encoding for files indexed by zebra. However, if records
391 themselves have an XML header that specifies and encoding that'll be used
394 XML filter (-t grs.xml).
396 Multiple registers. New setting in resource 'root' that holds base
397 directory for register(s). A group a databases may be put in separate
398 register in directory root/reg by using db name 'reg/db1' ... 'reg/dbN'.
402 Fixes for Digital Unix
404 Implemented hits per term using USR:SearchResult-1.
406 New Zebra API. Locking system re-implemented.
408 --- 1.1.stable 2002/02/20
410 Rank weight can be controlled with attribute type 9. Default
411 value is 34. Recommended values between 1-36.
415 Updated for YAZ version 1.8.
417 Added support for termsets - a result set of terms matching
418 a given query. For @attr 8=<set> creates termset named <set>.
420 Added support for raw retrieval. Element Set Name R forces the
421 text filter which returns the record in its original form.
423 Added numerical sort - triggered by structure=numeric (4=109).
425 Remote record import using Z39.50 Extended Services and Segments.
427 Fixed bug where updating a database with user-defined attributes
428 could corrupt the register (bad storeKeys).
430 Multi-threaded version.
432 Fixed bug regarding proximity.
434 Documentation updates.
436 Fixed bug in record retrieval module that occured on 64-bit OSF
441 Fixed bug in makefile for WIN32.
443 Fixed bug in configure script - used bash-specific features.
447 Added support for multiple records in one file for filter grs.sgml.
449 Changed record index structure. New layout is incompatible with
450 previous releases. Added setting "recordcompression" to control
451 compression of records. Possible values are "none" (no
452 compression) and bzip2 (compression using libbz2).
454 Added XML transfer syntax support for retrieval of structured records.
455 Schema in CompSpec is recognised in retrieval of structured records.
457 Changed Tcl record filter so that it attemps to read <filt>.tflt. If
458 that fails, the filter reads the file <filt>.flt (regx style filter).
460 Implemented new Tcl record filter - use grs.tcl.<filter> to enable it.
461 Zebra's configure script automatically attempts to locate Tcl. For
462 manual Tcl configuration use option --with-tclconfig=<path> to specify
463 where Tcl's library files are located.
465 Implemented "compression" of Dictionary and ISAM system. Dictionary
468 Added "tagsysno" directive to zebra.cfg to control under which tag the
469 system ID is placed. Use tagsysno: 0 to disable Zebra's system number
472 Added "tagrank" as above.
474 Changed file naming scheme for register files from <name>.mf.<no> to
477 Implemented "position"-flag for register type (as defined in
478 default.idx). When set to zero no position (or seqence number) is
479 saved in register for each word occurrence, thus saving some register
482 Implemented database mapping. Using mapdb one can specify a database
483 to be mapped to one or more physical databases. Usage:
484 mapdb <fromdb> <todb> ..
486 Added SOIF-filter. Thanks to Peter Valkenburg.
488 For the regx-filter "end element -record" may trigger a mark-of-record
489 if outer level is reached.
491 Tag sets may be typed in the reference to it. From the .abs-file the
492 "tagset" directive takes a third optional integer type for the tag set
493 referenced. From a .tag-file the "include" directive takes a third
494 optional type as well. The old "type" directive in the tag set itself
495 is still recognized but acts as the default type for the tag set.
497 Zebra supports the specification of arbitrary attributes sets, schemas
498 and tag sets, because of the change in YAZ' OID management system.
500 Fixed bug in Sort that caused it NOT to use character mapping as it
503 Zebra now uses GNU configure to generate Makefile(s).
505 Added un-optimised support for left and left/right truncation attributes.
507 Added support for relational operators on text when using RPN queries.
509 Added support for sort specifications in RPN queries. Type 7 specifies
510 'sort' where value 1=ascending, value 2=descending. The use attribute
511 specifies the field criteria as usual. The term specifies priority
512 where 0=first, 1=second, ...
514 Changed the way use attributes are specified in the recordId
517 Maximum number of databases in one Zebra register increased.
519 New setting, databasePath, which specifies that first directory during
520 update traversal is the database name (instead of a fixed one).
522 New setting, explainDatabase, which specifies that databases are
525 Modified Zebra so that it works with ASN.1 compiled code for YAZ.
527 Implemented EXPLAIN database maintenance. Zebra automatically
528 generate - and update CategoryList, TargetInfo, DatabaseInfo,
529 AttributeSetInfo and AttributeDetails records at this stage. The
530 records may be transferred as GRS-1, SUTRS or Explain.
532 Fixed register spec so that colon isn't treated as size separator
533 unless followed by [0-9+-] in order to allow DOS drive specifications.
535 Fixed two bugs in ISAMC system.
537 Changed the way Zebra keeps its maintenance information about attribute
538 sets, available attributes, etc.. Records in "SGML" notation using an
539 EXPLAIN schema is now used when appropriate.
541 Bug fix: Index didn't handle update/insert/delete of the same record
542 (i.e. same recordId) in one run (one invocation of zebraidx). Only the
543 first occurence of a record is considered.
545 Most searches now return correct number of hits.
547 New modular ranking system. Interested programmers are encouraged to
548 inspect rank1.c and improve the algorithm.
550 Bug fix: Lock files weren't removed as they should on NT.
552 Implemented Z39.50 Sort. Zebra's sort handler uses use attributes to
553 specify a "sort register". Refer to the gils sample records which refer
554 to index type "s" which is specified as "sort" in the default.idx file.
555 Each sort criteria can either be Ascending or Descending and at most
556 three sort elements can be specified.
558 Bug fix: Character mapping didn't work for text files.
562 Simple ranked searches now return correct number of hits.
564 The test option (-s) only makes a read-lock on the index as well
565 as using read-only operations anywhere.
567 Moved towards generic character mapping. Configuration file default.idx
568 specifies character map files for register types w, p, u, etc.
570 Implemented "begin variant" for the sgml.regx - filter.
572 Fixed a few memory leaks.
574 Added support for C++, headers uses extern "C" for public definitions.
576 Bug fix: The show records facility (-s) only displayed information for
577 the first record in a file (and not for every record in the file).
579 Added option "-f <n>" to limit the logging of record operations. After
580 <n> records has been processed no logging is performed (unless errors
583 Bug fix: the compressed ISAM system didn't handle update operations
586 Added setting, "maxResultSetSize", to hold the number of records to
587 save in a result set.
589 Bug fix: Complete phrase did't work for search operations.
591 Bug fix: temporary result sets weren't deleted.
593 Reduced disk space for saved keys (storeKeys = 1).
595 Added optional, physical ANY (key replication)
597 Implemented proximity operator in search.
599 Bug fix: the path name buffers used by file match traversal routines
600 have been extended to support long file names.
602 New C(ompressed) ISAM system. To enable it, specify "isam: c" in the
603 configuration file. The resulting register without "storeKeys" is about
604 half the size, and the memory used by zebraidx during phase 2 (merge) is
605 reduced to a minimum.
607 Reworked the way Regexp-2 queries with error tolerance are handled and
608 specified. The documentation has been updated accordingly.
610 Bug fix: Zebrasrv didn't search correctly when queries contained masking
611 characters. This bug was introduced in 1.0a8.
613 Zebrasrv now tag records with the proper database name.
615 New settings, memMax and keyTmpDir.
617 Changed name of setting lockDir (previously called lockPath) and
618 setTmpDir (previously called tempSetPath).
620 Generalized and changed record type specifications. In short, there are:
622 grs.sgml structured, "SGML-like" syntax
623 grs.regx.<filter> structured, Regular expression filter
624 grs.marc.<abs> Reads *MARC records in the ISO2709 format. <abs>
625 is the name of an abstract syntax file.
626 Bug fix: Result sets weren't sorted in operations involving boolean
627 operations with "ranked" operands.
631 Added national character-handling subsystem.
635 Small modifications to input filters and profiles.
637 Added support for SOIF syntax (with private OID).
641 Fixed buffer-size problem in indexing.
643 Added compression to temporary files for updating.
645 Added phrase registers.
647 Added dynamic mapping of search attribute to multiple termlists (ANY).
649 Scan support in multiple databases/registers.
651 Configuration settings are case-insensitive and single dash (-)
652 characters are ignored in comparisons.
654 The index processing ignores empty files - warning given.
656 New option to zebraidx (-V) displays version information.
660 Fixed problem in file-update system.
662 Fixed problem in shadow system; register was sometimes corrupted after
667 Fixed problems in the ISAM subsystem. Caused difficulties when updating
670 Fixed small problem in SUTRS-filter. A newline was sometimes inserted before
671 the rank and record number.
673 Fixed bug in the isam subsystem - caused a malfunction when accessing
674 words which occurred more than 10000 times.
676 Distribution should now include YAZ (Z39.50 protocol stack) to simplify
679 Server can now run under inetd. Use option -i, and -w <directory> to
680 set working directory to desired location.
682 New zebraidx command: clean - removes temporary shadow files.
684 Fixed bug in ISAM system. Occurred rarely during register updates.
686 Logging during index merge phase is improved. The remaining running
689 Temporary files generated by zebraidx are removed after each run.
691 Bug fix: Dictionary didn't handle 8-bit characters correctly; was obvious
692 when doing scan operations in dictionaries with European characters.
696 A whole slew of updates, to make the first publicized release. Get the doc
701 Memory-problems in ISAM fixed. More blocktypes added to the default setup
702 to increase performance on larger databases.
704 Various minor changes in data management system.
708 A couple of portability-problems resolved.
710 Changed some malloc() to xmalloc().