X-Git-Url: http://git.indexdata.com/?p=yaz-moved-to-github.git;a=blobdiff_plain;f=NEWS;h=ce5abab95ae4a01411ca77da3cbaa0d0d6f24988;hp=8fec5426746a8865c6c7b932d5847028e3123569;hb=1770f18530a45da520e9b47fedee3203d01e04af;hpb=796a205f03250e4ed1eb4986f3e7a0feb36dd2bb diff --git a/NEWS b/NEWS index 8fec542..ce5abab 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,549 @@ +--- 4.2.56 2013/04/26 + +Fix yaz-client connect (crash in 4.2.55). + +--- 4.2.55 2013/04/26 + +Fix proxy/host problem for ZOOM that could occur if unix sockets +was in use. + +New Debian package libyaz4-dbg. + +Fix problem with cs_close hang/wait in SSL mode. Invoke gnutls_bye +differently in tcpip_close. + +--- 4.2.54 2013/04/17 + +Windows installer of YAZ includes YAZ4J as a component. + +CQL: allow "extra terms" again, by default. Thus, the CQL parser +accepts a superset the standard. This reverts to CQL as of YAZ 4.2.51 +and earlier. CQL parser may operate in strict mode where queries +not conforming to the standard BNF are rejected (syntax error). +Whether the CQL parser operates in strict mode is controlled with +cql_parser_strict . Strict mode enabled with option -S for utilities +cql2pqf and cql2xcql . + +yaz-ztest: returns OPACXML records for syntax=XML and element set=OP. + +--- 4.2.53 2013/04/10 + +New charset, MARC-8c, which is like MARC-8 but allows control characters +0x01 through 0x1F, except ESC to be passed through verbatim. + +Fix a problem with duplicate sub field character in MARC output. Problem +happened only when output was MARC-8 encoded. + +--- 4.2.52 2013/04/03 + +yaz-ztest: Use hash for fake hit count. Before, random number was used. +This is problematic for unit tests using yaz-ztest. This patch also makes +a hash for CQL queries. For RPN, any term with a leading digit will be +treated as pseudu hit count (to force any particular hit count by a client). + +yaz-ztest: return unique MARC dummy records. Beyond record offset +24, the author surname is patched with offset to make all records +unique. + +COMSTACK: make @ bind to IPV6+IPV4 consistenly and avoid depending +on system wide default (/proc/sys/net/ipv6/bindv6only). +@4 binds to IPV4 only. @6 binds to IPV6 only. + +Update CQL API documentation a bit + +CQL: don't accept extra terms for queries. Makes the CQL parser +stricly conformant to the standard. + +--- 4.2.51 2013/02/12 + +GFS: scan handler gets extra_args from request and may return extra +response data . That's for SRU only. + +Solr scan support for ZOOM and yaz-client. Patch from Simon Jacob. + +--- 4.2.50 2013/01/30 + +New function, yaz_xml_to_opac, to convert from OPACXML to Z39.50 OPAC. + +OPACXML: sync spelling of availabilityDate in XML to that of Schema +opaxxml.xsd. The schema had the correct spelling of this member always. +The ASN.1 was mis-spelled and still is (API change) - we are +not changing this. However, the OPAC to XML conversion had yet +*another* typo. + +GFS: Allow conversion between Z39.50 OPAC and OPACXML via retrieval +element. + +New function, z_ext_record_oid_nmem, which is similar to +z_ext_record_oid but takes NMEM rather than ODR + +--- 4.2.49 2013/01/18 + +ICU: New conversion element 'join' which joins tokens into one +with a custom character given by rule attribute. + +--- 4.2.48 2013/01/10 + +New GFS search member: present_number. This is a hint to the +search handler how many records are going to be fetched immediately +following search. There's no guarantee that it's going to be the case, +since it depends on hit count (for Z39.50 piggyback) and whether errors +turn up. + +Make a few functions static (private). The functions getbyte_stream, +ungetbyte_stream and yaz_gets are now private. They were never declared +in a header file. + +yaz-marcdump: exit(5) for serious decoding errors + +ISO2709 decoding: be more picky WRT header + +GFS: bail out of fetch if last_in_set is set. Until now, last_in_set, +was only used as to indicate that next-result-set-position should be +set to zero. +http://www.loc.gov/z3950/agency/markup/04.html#Next-result-set-position +Based on patch from Ashley Sanders. + +Make some functions static/private in PQF parsing. Those functions were +never declared in a public header and so should not be in use by +applications based on YAZ. + +--- 4.2.47 2012/11/23 + +ZOOM: special options to control APDU logging. +Using ZOOM_connection_option_set(c, "saveAPDU", "1") will enable +logging of APDUs until "saveAPDU" option is set again. If logging is +already enabled the effect is that current APDU buffer is cleared. +Using ZOOM_connection_option_set(c, "saveAPDU", "0") disables +logging (default behavior on new connection). +The APDUs sent/received can be obtained by using +Using ZOOM_connection_option_get(c, "APDU") or +ZOOM_connection_option_getl(c, "APDU", &l). + +GFS: relay implementation_version from backend +From 3.0.40 and later, implementation_version from backend was +not used in init response at all. + +--- 4.2.46 2012/11/15 + +Fix cs_rcvconnect for SSL comstack GNUTLS. Did not work in non-blocking +mode at all. Probably never did. + +Make a new version of ztest.pem that, at least, is not expired. + +--- 4.2.45 2012/11/13 + +Fix description of CCL directive 'case'. + +More tight checking of form arguments (yaz_uri_to_array). Makes +YAZ better at rejecting big and/or bad POSTs. + +SRU decoding does not report more than up to 10 invalid arguments. + +rpn2cql: less quotes in generated terms. Semantics unchanged, but +it's prettier this way. + +--- 4.2.44 2012/10/25 + +New utility yaz_decode_init_diag. Defined in proto.h. + +Allow HTTP protocol on unix local socket. Also Z39.50 database may be +given for unix local socket. + The COMSTACK string for doing so is: + unix:: + Examples: + unix:/myfile:tcp:localhost:9999/mybase + unix:/myfile:localhost:9999/mybase + unix:/myfile:http://fakehost/mybase + +debian: Use dpkg-buildflags in build process. + +--- 4.2.43 2012/10/19 + +ZOOM C: don't trust recordPosition always. Some servers, such as IGI +global's SRU service uses a recordPosition scheme starting from zero. +First record's position should be one +according to the standard. + +--- 4.2.42 2012/10/18 + +Allow more namespaces for SRU. This is to allow, for example, an +incorrect namespace returned from server +http://services.igi-global.com/sru . + +Minor fixes for compilation on Windows. + +--- 4.2.41 2012/10/11 + +daemon: log total number of bad signals from child on stop so that it +is more easy to spot problems with child worker processes. + +retrieval: empty backend name attribute omits element set +from being used in retrieval. + +--- 4.2.40 2012/10/04 + +Implement Solr sort strategy "solr", similar to sru11 sorting. +This will implement the correct parameters on Solr requst to do +native sorting. + +--- 4.2.39 2012/09/28 + +Implement ccl_qual_fitem2. Like ccl_qual_fitem but with diagnostic +string returned. + +MARC: Extra/missing indicator handled for MARCXML. +The indicator length is given in header and is the same for the +whole record. If attributes ind1, ind2 are missing, then content +character defaults to blank. Extra indicator attributes (beyond) +indicator length are treated as an error. + +Reformat: delete trailing whitespace + +--- 4.2.38 2012/09/17 + +MARC: fix non-ASCII characters for leader. This is to ensure we can +generate XML in UTF-8. + +CCL: new special attribute s=ag . Almost s=pw,al but preserves words +in one node with structure word. + +--- 4.2.37 2012/09/04 + +yaz-json-parse: JSON output may be pretty printed by using option +-p twice. + +Increase default value of max message for Z39.50 Init for GFS, ZOOM +and yaz-client. Old value was 1 MB. New value is 64 MB. + +--- 4.2.36 2012/08/22 + +yaz-client: don't display bogus records from SRU servers. Some SRU servers +return more records than given by *maximumRecords*. Skipping those in +display! + +record_render: simplify buffer return handling and fix at least one bad +memory reference, that broke the test_embed_record sometimes. + +Allow http://user:pass@host notation for yaz_url_exec. + +zoomsh: allow shell command to be executed. Via the system(3) call. + +yaz_poll: Allow num_fds == 0 + +yaz-client: fix bad encoding of "itemorder item". The itemorder +item - command forgot to use length of encoded Item Request +package. Error present since YAZ 3. + +yaz-ztest: fix bad usage of session handle. Bug was introduced when +result sets were saved (271a9c83). + +SRU recordPacking: guess record packing in response rather than looking +at the recordPacking element which, OF COURSE, changed semantics in SRU +2.0. + +Decode SRU 2.0 responses. The official namespace for responses, according +to http://docs.oasis-open.org/search-ws/searchRetrieve/v1.0/cs01/schemas/sruRes +appears to be http://docs.oasis-open.org/ns/search-ws/sruResponse . +nature.com's server, however, usee namespace +http://docs.oasis-open.org/ns/search-ws/sru-2-0-response . For this reason +we are now using a glob-expr to match up the namespace. ZOOM C and +yaz-client updated. Perhaps SRU 1.2 uses same namespace, I don't know. +I can not find a single real example showing the "real" namespace in +the documentation from OASIS. The namespace for requests is different +from responses, but this is not added to YAZ yet, when we don't encode +them yet. + +--- 4.2.35 2012/06/26 + +zoom/record_render: add support for Base64 decoding fo MARC records +within XML element. For example: "xml; charset=marc-8; base64=/my/text()" +which will decode base64 + ISO2709 marc-8 encoded material from text node +under /my . + +--- 4.2.34 2012/06/07 + +Add missing definitions ZOOM_resultset_get_facet_field_by_index +and ZOOM_resultset_facets_names. + +zoom: fix bad free for ZOOM_connect - happened when ZOOM connection +options was in use. + +Change APDU dump printing to avoid long leading blanks. For levels 16 +or larger the level is shown and the indentation is truncated but +recursed with modula 8. This makes it still reasonable easy to read +- even for nested BER dumps. + +Fix bug completeBER_n WRT too nested BER. Function completeBER_n could +return 0 (incomplete package) in case where the BER package was +considered too nested (return value of -2). + +dumpber: lines of level 15 are not indented further. Two spaces per +indentation; indentation 15 or more is indented to a fixed indentation +and augmented with a level= indication. + +zoomsh: increase max size of command+args + +GFS: No limit in RPN buf size in CQL to RPN conversion. + +--- 4.2.33 2012/05/09 + +New function for daemon utility, yaz_daemon_stop . + +Makes yaz_srw_sortkeys_to_sort_spec public on Windows. + +--- 4.2.32 2012/04/25 + +Add some Bib-1/SRU diagnostics + mappings. Bib-1: 1073 and +Bib-1 1074: Proxy failure. SRU: 1074: Proxy failure + +--- 4.2.31 2012/04/23 + +YAZ daemon utility relays SIGUSR1 to child process. + +ZOOM: handle trailing slash for Z39.50 urls. Previously localhost:9999/ +would result in en empty database list - which in turn would result in +encoding failure for search. + +Fix reading beyond string in yaz_sort_spec. + +--- 4.2.30 2012/04/11 + +Add two new SRU diagnostics: + 236: Access to specified database denied + 1015: Init/AC: Maximum number of simultaneous sessions for Userid + +Document the very useful ZOOM-C option "apdulog". + +CQL to RPN: term may be passed through verbatim. This is enabled with +truncation.cql . The assumption being that the term is already CQL. + +--- 4.2.29 2012/03/20 + +record_conv: leader spec may be given for marc element. +The leader spec is given as attribute leaderspec for element marc. +For example to convert a MARC21 record from MARC-8 to UTF-8: + + +record_conv: allow user-XSLT params. For example: + + + + + +record_conv: fix error handling for missing marc charsets. + +CQL: fix extra terms in wrong order. + +--- 4.2.28 2012/03/06 + +record_conv: modify new API for conversion types. The conversion types +may be passed in new functions yaz_record_conv_configure_t and +yaz_retrieval_configure_t. Removing yaz_record_conv_add_type . + +Avoid mixed statement/variable declare; fixes compilation with Visual +Studio. + +--- 4.2.27 2012/02/28 + +Add non-standard SRU diagnostic: 235: "Database does not exist". +Couldn't live without it. HTTP 404: "Not found" is just so anonymous. +Update SRU to Bib-1 diagnostic maps accordingly. + +CCL: add two new diagnostics for unsupported masking. The new +diagnostics are: "Embedded truncation not supported" and +"Single charcacter mask not supported". + +--- 4.2.26 2012/02/23 + +CCL: fix use of "term" field in sub queries. Bug introduced in YAZ 4.2.25. + +--- 4.2.25 2012/01/11 + +Build YAZ for Ubuntu Oneiric. + +CCL: allow parantheses for proximity operands. For example CCL query +(a) % (b) is now allowed. + +GFS: fix bend_assoc_is_alive. When cs_get returns 0 the association is +marked dead, rather then being destroyed. If not, the session memory is +being free'd before the search/scan handler is finished. Also destroy +'request' memory in request_release rather than in various places, +which resulted in leaks from time to time. + +--- 4.2.24 2011/12/21 + +cql_sortby_to_sortkeys: honor sort prefix. + +RPN to CQL: always quote resulting terms. For bizarre reasons, JSTOR's +SRU server returns 0 hits on ISSN searches unless a CQL term is quoted. +The rpn2cql family of functions now always returns quoted CQL terms. + +Fix CQL to CCL conversion for multiple terms. Add two test cases as well. + +--- 4.2.23 2011/12/15 + +Allow =val to be omitted for yaz_uri_to_array. Also, multiple &'s in +sequence is treated as one. For example, &&a=b&&&c& is identical to +a=b&c= . + +--- 4.2.22 2011/12/05 + +Add ZOOM_connection_save_apdu_wrbuf. This function may be used to save +APDUs for a connection to a WRBUF. + +--- 4.2.21 2011/11/23 + +Fix cql_sortby_to_sortkeys. Completely broken before. + +Make CCL's @truncation work again. + +--- 4.2.20 2011/11/09 + +New ZOOM connection setting: tproxy. It's like proxy, but does not +set Z39.50 proxy OID or change HTTP for proxy behavior. Can be used +to set resolved address + port for a connection outside ZOOM itself. + +Fix atoi_n_check digit test. Thanks to Giannis Kosmas for spotting it. + +--- 4.2.19 2011/11/07 + +Fix uninitialized variable in Solr spell support (could cause SEGV +for SRU response decoding). + +Error handling for Z_APDU_close uses ZOOM_set_dset_error() to pass +back-end's addinfo through into the ZOOM response addinfo. + +When ZOOM_set_dset_error() is passed an addinfo2 as well as addinfo, +they are separated by a colon and a space rather than simply +concatenated. + +--- 4.2.18 2011/10/28 + +Add support for Solr spell-checking/suggestions through ZOOM. +If a Solr target has been configured correctly for spelling-checking +and suggestions, these will now be returned on the ResultSet option +"suggestions". zoomsh has been extended to show this with the cmd suggestions. + +--- 4.2.17 2011/09/28 + +Fix function nmem_strsplit_escape2 which extends nmem_strsplit_escape +with a flag whether to substitute escape sequences or not. + +--- 4.2.16 2011/09/22 + +Fix nmem_strsplit_escape to substitute escape char properly. + +New option for zoomsh -e , which makes it abort on first error and +return process exit code to indicate success/failure. + +YAZ options utility supports special option -- which makes all it treat +all following arguments as non-options. + +--- 4.2.15 2011/09/16 + +Fix yaz_file_path_resolve/yaz_path_fopen/yaz_fopen so that path is not +searched if absolute filename is given. + +Fix conv_xslt() error-reporting when stylesheet is not found. +(The value of fullpath is meaningless in this situation.) + +--- 4.2.14 2011/09/07 + +Fix problem in CQL sort node creation: un-initialized memory. + +New ZOOM utility ZOOM_query_sortby2. That takes a strategy parameter: one of +"z3950", "type7", "cql", "sru11" or "embed". The "embed" chooses type-7 or +CQL sortby depending on whether Type-1 or CQL is actually sent to the +target. + +New sortspec utilities: yaz_srw_sortkeys_to_sort_spec, +yaz_sort_spec_to_srw_sortkeys, yaz_sort_spec_to_type and 7yaz_sort_spec_to_cql. + +rpn2solr supports Z39.58 truncation (104). Thus, CCL truncation mode t=z +will make it possible to use both single-character mask (#) and wildcard (?) +in CCL . Throw diagnostic for left truncation because that is not supported +by Solr. + +--- 4.2.13 2011/09/01 + +Fixes for cql2rpn, cql2ccl and ccl2rpn; mostly in the area of special +cases using escapes (\-sequences) and masking. + +PQF parse fix: A trailing backslash followed by end-of-string (\0) +would result in a read one byte beyond string. + +rpn2ccl may use Z39.50 masking for terms (t=z). + +ICU check fix. Don't include library -lCrun on Solaris, unless present. + +ICU check fix: Use --cppflags-searchpath to determine ICU include files; +not --cppflags (which seeems like an error in icu-config). + +--- 4.2.12 2011/08/27 RPM package only + +cql2ccl: fix conversion of terms: operator characters were not +quoted/escaped. + +--- 4.2.11 2011/08/25 RPM package only + +ccl2rpn: Conversion to regexp-1 terms (trunc=102). For mode t=x, the +CCL parser will map both # and ? to their regular expression +quivalents (. and .*). + +rpn2cql: deal with truncation=102 (regexp-1). And ensure proper quoting +of reserved CQL characters. + +cql2ccl: deal with both * and ? in conversion. Also leave escaped ones as +is, so that they are passed through. + +--- 4.2.10 2011/08/24 + +ZOOM SRU mode: more aggressive re-connect. + +Private function prox_unit_name() renamed to yaz_prox_unit_name() and +made available as a utility function, declared in + +rpn2cql_structure() supports proximity nodes (except that the Type-1 +proximity node's "exclusion" binary is ignored, as there is no way to +express that in CQL). Fixes bug #4564. + +--- 4.2.9 2011/08/18 + +Fix SEGV that could occur if set command of zoomsh was given without +value (2nd argument). + +Fix SEGV that could occur if unsupported transport was given for +COMSTACK - say https (when SSL was unavailable). + +Build yaz-url on Windows too. + +--- 4.2.8 2011/08/11 + +New utility to get web content. Header is url.h. A command line tool +yaz-url is part of the distribution and packages. + +Fixes for SSL COMSTACK (when gnutls is in use). + +New functions yaz_base64{encode,decode}. Header is base64.h. + +--- 4.2.7 2011/08/03 + +ZOOM C includes HTTP response buffer as additional info if it fails +to decode the SRU response. + +Make wrbuf_json_puts available. + +--- 4.2.6 2011/07/28 + +zoom: throw diagnostic for invalid value for option "sru". Before a +bad SRU request was sent. + +Adding new utility wrbuf_vp_puts. + +Add cclxmlconfig.obj to win/makefile. It was missing completely. + --- 4.2.5 2011/07/21 ZOOM HTTP: use absoluteURI's in proxy mode as required by RFC 2616 (HTTP). @@ -14,7 +560,7 @@ modified with a new root element if the recordData contains multiple children elements. This is to make the record XML parseable afterwards. -RPN to SOLR conversion: encode special characters. +RPN to Solr conversion: encode special characters. Generic Frontend Server: fix uinitialized member, stepSize. @@ -116,7 +662,7 @@ so it is appropriate to set Present Status to partial-4. Move yaz-config man pages to section 1. -More documentation about SOLR. +More documentation about Solr. tcp comstack: change behavior for @-specs (anyaddr). @ listens on AF_INET (IPV4) peers, and @6 listens to AF_INET6 (ipv6) peers. @@ -135,7 +681,7 @@ Fix yaz-config output: echo_source not set correctly by configure. --- 4.1.0 2010/10/05 -SOLR WebService support for yaz-client and ZOOM. +Solr Web Service support for yaz-client and ZOOM. Define record syntax JSON. OID: 1.2.840.10003.5.1000.81.3 .