X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=NEWS;h=b04ef84eb78ab2166bd049bf32a307ff09824781;hb=15b65f7c9769142b709d079496929727d41a4229;hp=57cf55d7b6eb87b35784ea37b1d8715aae0ce5fd;hpb=b2a58f35bdd9d45607f5b226743df8983c532433;p=metaproxy-moved-to-github.git

diff --git a/NEWS b/NEWS
index 57cf55d..b04ef84 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,704 @@
+--- 1.4.0 2013/10/04
+
+Build RPM/Deb packages with YAZ 5 and yazpp 1.5. Metaproxy still compiles
+with older version from source, but packages now use YAZ 5 that supports
+SRU 2.0.
+
+sru_z3950: Fix SEGV for decoding invalid SRU requests
+Incorrect reference to union (was explain response rather than
+search response).
+
+session_shared: match additionalSearchInfo (includes facets). In
+layman's terms means that means that facets are properly handled
+by this filter. Eg a change in requested facets will issue new
+search and facets from exisiting result set are cached.
+
+--- 1.3.62 2013/09/11
+
+http_rewrite: better handling of <script> content in HTML parser. MP-486
+
+http_rewrite: fix threading issue with regex. MP-482
+
+bounce: don't use strdup for fixed length content. Error occurred in echo
+mode only.
+
+session_shared: expire backend class when no instances left. This
+means memory clean up, but not really a leak fix.
+
+virt_db: fix leak for naked ptr.
+
+cql_rpn: use config relative path for cql2rpn file. MP-481
+
+Updates for upcoming YAZ 5.
+
+--- 1.3.61 2013/08/29
+
+New configuration directories ports.d/routes.d for metaproxy
+configuration. This allows 3rd party packages to install config
+for ports and new routes. MP-477
+
+--- 1.3.60 2013/08/21
+
+Fix metaproxy-config --libs to return non-mt over mt libs.
+The -mt versions do not exist as shared objects and cause
+linker errors for applications linking with Metaproxy.
+Problem present in versions 1.3.56 - 1.3.59. MP-475
+
+HTML: clean up char checks. Add ptr to HTML parse spec
+
+HTML parser stops eating tag when <-char is met  MP-473
+
+rewrite: do not even compare content types, if we don't have any
+the regex_match caused a segfault if we had no content-type in the
+headers. MP-471
+
+rewrite: within header=... matches only the header value
+Used to match the whole header line, but we need to use
+^$ anchoring for host-relative regexps, and they need to
+anchor at the ends of the value. MP-470.
+
+rewrite: Ruleset functional MP-469
+
+--- 1.3.59 2013/07/12
+
+session_shared: negotiate init sz parameters.
+By default, session_shared will honor the paramters given by client.
+An alternative, but not implemented, algorithm would be to only
+re-use sessions with same sz parameters. This however, could reduce
+re-use of sessions a lot. Hence this, simple algorithm instead. The
+negotiation to a maximum sz values can be enabled by using
+   <init maximum-record-size="sz1" preferred-message-size="sz2"/>
+in configuration. This will make session_shared send those values
+to backends and use returned init values in negotaition to
+client before session_shared.
+
+session_shared: resolve result-set references.
+The reference is substitued with the query for old set if database
+match and query type is Type-1. If not the reference will issue a
+diagnostic.
+
+sort filter: relays RecordComposition including elementSet in use for
+search.
+
+http_client: better diagnostics
+
+http_client: disables HTTP redirect handling by default.
+It can be configured with max-redirects in configuration.
+
+--- 1.3.58 2013/06/12
+
+Fix loading of filters cql_rpn and sru_z3950 . Was broken by 1.3.57.
+
+--- 1.3.57 2013/06/12
+
+Make RouterChain and RouterXML part of public API (libmetaproxy).
+
+http_client: honor X-Metaproxy-Proxy as an alternative to
+the "proxy" configuration. Allows proxy to be set per HTTP request.
+
+sru_z3950: For SRU requests recognize x-client prefixes as a way
+to ingest "database parameters".. The suffix of x-client- serves
+as database argument name. Multiple may be given. For example:
+  http://path/db/?operation=searchRetrieve&x-client-user=a&x-client-password=b
+is turned into Z39.50 database
+  db,user=a&password=b
+
+--- 1.3.56 2013/05/27
+
+New filter present_chunk. This filter allows splitting of present requests into smaller chunks.
+
+New filter sd_remove, which replaces surrogate diagnostics with
+SUTRS records. This filter tries to deal with clients that breaks
+when receiving surrogate diagnostics.
+
+new filter http_client. Same role as z3950_client, but just for
+HTTP. Allows Metaproxy to operate as a HTTP proxy.
+
+New filter http_rewrite1, which serves as purpose on how to rewrite
+HTTP content with Metaproxy.
+
+Use libboost-system if available, because newer versions of libboost-threads
+requires libboost-system.
+
+Allow frontend_net to serve SSL/HTTPS, because we can specify a server
+certificate now.
+
+--- 1.3.55 2013/04/10
+
+zoom: Torus setting: authenticationMode which may hold values "basic"
+and "url".
+
+--- 1.3.54 2013/03/27
+
+configure: check for Libxslt support in YAZ and rely on definitions
+from yaz-config, rather than doing own xslt/xml2 checks.
+
+Allow timestamp format for log lines to be given with option -m
+for metaproxy. Similar to yaz-ztest/GFS option -m.
+
+Document log category 'line' for log filter.
+
+Solr configuration example in etc/config-solr.xml
+
+multi: do not mess with request package. For scan and search,
+database-names were modified and the request package was left with
+dangling pointers that caused a log filter to fail (request package
+inspected after multi was invoked).
+
+Documentation: fix id of metaproxy-config ref .
+
+Documentation: update for USEMARCON support.
+
+--- 1.3.53 2013/01/31
+
+record_transform: allow OPACXML to OPAC conversion
+
+zoom: document content-proxy
+
+zoom: document db parm content-{user,password}, torus_url. Also
+<torus auth_url="..> which makes zoom operate in authenticated mode
+via Torus lookup.
+
+--- 1.3.52 2012/11/30
+
+zoom: fix http://-check. SRU search broken in 1.3.51.
+
+--- 1.3.51 2012/11/15
+
+zoom: only append http:// for SRU if not already given . Makes it
+possible to use https-URLs.
+
+Use yazpp 1.4 or later due to new ABI.
+
+frontend_net: allow max package size to be specified. Controlled by
+attribute 'max_recv_bytes' of element 'port'.
+
+--- 1.3.50 2012/10/29
+
+For most generated diagnostics by filters, typically temporary
+system error, augment addinfo message with filter name.
+
+Add support for CentOS 6 packages.
+
+sru_z3950: interpret Z39.50 diagnostics part of Init Response.
+
+Add man page for metaproxy-config.
+
+--- 1.3.49 2012/10/18
+
+zoom: new config element passthru. This defines an element set that is
+passed verbatim to the backend in pz2 mode.
+
+auth_simple: allow both idPass and open auth. Also fix SEGV that could
+occur for open auth.
+
+--- 1.3.48 2012/10/08
+
+z3950_client: fix string handling for init diagnostics in the case where
+addinfo was absent.
+
+query_rewrite: no charset conversion for error queries. Avoid performing
+charset conversion for query conversion errors. The resulting Type-1
+query is invalid in this case.
+
+--- 1.3.47 2012/10/04
+
+For graceful stop (SIGUSR1), metaproxy will kill children processes.
+
+Log when metaproxy is invoked in test mode (-t).
+
+Log PID for each log message.
+
+multi: combine both NSDs and multiple NSDs into one (diagnostics).
+
+z3950_client: fixup multiple NSD's too. Like surrogate diagnostics,
+a (backend=target) is appended for each diagnostic so that can be
+determined the origin of error.
+
+multi: hideerrors also in use for present response.
+
+Solr example in zoom man page.
+
+Fix typo in query_rewrite config example.
+
+Log at least first non surrogate diagnostic. Instead of
+"Z_Records_multipleNSD"
+
+multi: Fix check for opt Records in present response. A SEGV could occur
+if a target did not return records and no diagnostics.
+
+--- 1.3.46 2012/09/24
+
+Yet another fix for graceful stop.
+
+--- 1.3.45 2012/09/12
+
+sru_z3950: relay surrogate diagnostics.
+
+--- 1.3.44 2012/09/10
+
+zoom: Honor extraArgs, strip # in zurl.
+
+multi: fix SEGV for scan. Reported by Rustam T. Usmanov.
+
+zoom: url_recipe allow %{var[reg]} substitutions.
+It's like ${var[reg]} but does URI component encoding as well.
+
+--- 1.3.43 2012/08/22
+
+RPM/Deb: require yazpp 1.3.2, because it fixes a missing close of
+non-keepalive HTTP sessions.
+
+http_file: allow zero size files (bug fix)
+
+Fix zoom schema: auth_url is optional
+
+--- 1.3.42 2012/08/20
+
+zoom: fix un-init memory (could result in SEGV at session close)
+
+frontend_net: handle X-Forwarded-For. This is stored in package.origin()
+and may be used in filters as usual.
+
+zoom: authentication via Torus lookup
+Enabled by new configuration setting: auth_url . If this is defined,
+the zoom module will contact Torus for auth/IP lookup and that will
+define realm to be used in session.
+
+--- 1.3.41 2012/08/10
+
+virt_db: Fix database name - first character we removed by mistake.
+Bug introduced in 1.3.40.
+
+--- 1.3.40 2012/08/06
+
+virt_db: Don't use "unix path" as database for Unix sockets.
+
+--- 1.3.39 2012/08/02
+
+z3950_client: Don't tamper with / in zurl - fixes problems
+using Unix sockets for backends.
+
+session_shared: fix integer overflow problem with hit count.
+
+session_shared: avoid long usage of cached queries.
+
+zoom: New database argument , torus_url to set URL of Torus
+record to be fetched.
+
+--- 1.3.38 2012/07/04
+
+Fixes for 'metaproxy reload' for RPM/Debian packages. 
+
+zoom: Allow rpn2cql conversion to be augmented with a properties file.
+Element rpn2cql can be specified per-target for the zoom filter.
+
+log: New category "line" which is a single-line log message with
+most important parts of a HTTP/Z39.50 request/response.
+
+--- 1.3.37 2012/06/01
+
+frontend_net: add facility to report live statistics about response
+times and current thread usage. The report is an XML document
+and is triggered for a specific URL path. This is set by <stat-req>
+in the configuration of frontend_net. By default this report is
+disabled (same as empty value for <stat-req>).
+
+Fix reload problem in init.d script for RPM/Debian. 
+The '/etc/init.d/metaproxy reload' in some cases would stop but not
+start metaproxy - effectively stopping the service.
+
+Don't display start log message when config testing (-t).
+
+zoom: avoid CCL message limit of 1K in ZOOM filter.
+
+--- 1.3.36 2012/05/10
+
+metaproxy init script reacts to reload which does graceful stop +
+start and is used by logrorate script.
+
+--- 1.3.35 2012/05/09
+
+Fixes for Windows compilation.
+
+Metaproxy daemon reacts to SIGUSR1 in which case it will leave
+existing sessions running but terminate the monitor process that is
+used in keepalive mode.
+
+Change start log message to 'metaproxy start'. Was 'Metaproxy start'
+before. Consistent with 'metaproxy stop' message.
+
+zoom: document zoom's proxy_timeout
+
+z3950_client: fixup addinfo for init response
+
+virt_db: return other diagnostic if init is rejected. Diagnostic 236:
+"Access to specified database denied" is returned, but only if init
+response from backend does not carry diagnostics already.
+
+multi: relays user info for first rejected target. The other-information
+field for init response for first rejected target is in common init
+response. This ensures that some backend init diagnostics are returned to
+the frontend.
+
+--- 1.3.34 2012/04/25
+
+frontend_log: extend logging (when message is given in config).
+
+--- 1.3.33 2012/04/25
+
+zoom: Fix check for proxy. Use diagnostic 1074 for proxy failures.
+
+--- 1.3.32 2012/04/24
+
+sru_z3950 filter passes through incoming Z39.50 requests again. Was
+not handled by mistake in Metaproxy versions 1.3.29, 1.3.30 and 1.3.31.
+
+--- 1.3.31 2012/04/24
+
+zoom: fixes for retry/proxy failover logic. More logging.
+
+--- 1.3.30 2012/04/23
+
+zoom: return addinfo="proxy failure" for proxy failures. This also
+changes the scheme for dealing with proxies a bit.  zoom now always
+do proxy check, even if there's no failover. Fortunately, that's not
+very expensive.
+
+zoom: Init rejected wo diagnostics maps to Bib-1:1014.
+Bib-1 1014, "Init/AC: Authentication System error", seems pretty
+generic. We don't know why the init would fail; IP, user, block IP, etc.
+sru_z3950 filter maps this to SRU diagnostic 3: "Authentication error".
+
+zoom: guess if invalid SRU response was an auth failure
+
+session_shared: disable session retry by default . Can be enabled by
+restart="true" in resultset section.
+
+--- 1.3.29 2012/04/18
+
+zoom: may read cf-proxy's config settings.
+This reduces manual configuration of the content proxy server host.
+The ZOOM module will use the old style session.proxyhost if "server"
+is given (deprecated) and use the new style proxyhost/session if
+"config_file" is in use.
+
+frontend_net: remove incoming requests that can not be handled
+Requests in queue for filter frontend_net are removed if client
+closes connection for the session and request is not yet handled.
+
+sru_z3950: serialize requests. This is to ensure that pipelined
+HTTP requests are handled properly.
+
+--- 1.3.28 2012/04/04
+
+New class, wrbuf, that wraps YAZ' WRBUF.
+
+--- 1.3.27 2012/03/30
+
+Metaproxy daemon is now running as user "metaproxy". This user is
+managed by package maintainer scripts. The user information is stored
+in /etc/metaproxy/metaproxy.user . 
+
+--- 1.3.26 2012/03/20
+
+New filter, sort, that allows sorting of XML records via Z39.50
+retrieval.
+
+Filter record_transform: USEMarcon support. For example:
+  <retrieval syntax="usmarc">
+     <backend syntax="unimarc" name="F">
+       <usemarcon stage1="/etc/usemarcon/uni2us/uni2us.ini"/>
+     </backend>
+  </retrieval>
+
+Filter multi: empty route pattern uses route as default.
+  That is
+   <target route="z3950.indexdata.com">z3950.indexdata.com</route>
+  may use the shorter notation
+   <target route="z3950.indexdata.com"/>
+
+Filter query_rewrite: use config path to locate XSLT.
+
+Filter frontend_net:  log statistics.
+If a custom message element in frontend_net configuration it makes
+the frontend_net filter produce a log entry whenever an operation
+is completed. The log is always written using yaz_log with the
+custom message given, the package ID (session ID) , the time
+the operation has been active (end-to-end) , the threads currently
+active/total and the size of the request input/output queues.
+
+query_rewrite: charset conversion of query terms.
+This is an optional feature. Enabled by element <charset> in
+configuration with attribute 'from' and 'to' that specifies
+from/to encodings. The default 'from' encoding is UTF-8.
+
+Filter record_transform: honor piggyback searches.
+
+Filter zoom: re-connect when database is reused.
+Re-connect is necessary if a connection is lost or a timeout occurs.
+If not, we'll be seeing zero hits, when we shouldn't.
+
+--- 1.3.25 2012/02/29
+
+zoom: Change scheme for dealing with content proxy URLs. The module
+now has its own common XSL transform that is enabled by record_xsl
+attribute in the configuration. This XSL gets parameter cproxyhost
+that holds the host for the content proxy session . The genearted-url
+is created only if urlRecipe is enabled. In previous version default
+value of urlRecipe was ${md-electronic-url}; it is now empty and
+disabled by default. Also generated-url no longer includes the content
+proxy host. Content proxying URLs is supposed to the handle in the record_xsl
+stylesheet.
+
+zoom: new setting timeout that specifies timeout for outgoing Z39.50/SRU
+connections. Default value is 40 (seconds).
+
+zoom: tune CCL to Z39.50 diagnostics mapping. Throw Bib-1: 108
+"Malformed query" for syntax errors and related ones; throw
+Bib-1: 3 "Unsupported search" for everything else. Details from
+the CCL parser is provided as additional-info.
+
+multi: new setting hideerrors that makes multi only relay diagnostics
+to a client if all backends fail.
+
+--- 1.3.24 2012/02/23
+
+zoom: empty ccl_map field-mappings disables CCL map.
+
+zoom: fix Explain search that could return error if session was
+re-used.
+
+--- 1.3.23 2012/02/10
+
+zoom: returns Explain records for database IR-Explain---1. The
+target profile records are converted to Explain records via XSLT.
+
+session_shared: make error recovery configurable. Whether a search
+is restarted on errors or not, is configurable by attribute "restart"
+in element "resultset" in configuration.
+
+session_shared: only relay init for first request. session_shared, for
+each backend class, now only relays first init request (rather than all).
+The module only needs to save init response info once. This avoids the
+need to check for session max.
+
+--- 1.3.22 2012/02/02
+
+zoom: fix memory leaks.
+
+session_shared: maximum number of sessons is configurable. Default
+value is 100. Result-sets for backends without named result sets are
+only re-used if databases is identical for a search. This is under the
+assumption that performance may suffer for those backends when
+changing the database for each search.
+
+multi: combines diagnostics from all backends. All diagnostics from
+backends are combined into multiple non-surrogate diagnostics; rather
+than just one diagnostic from first backend.
+
+--- 1.3.21 2011/12/27
+
+zoom: relay realm to cproxy parameter file.
+
+Configuration: Allow filter collection element inside route. This
+makes it possible to include multiple filters at once.
+
+zoom: nocproxy also disables urlRecipe.
+
+--- 1.3.20 2011/12/21
+
+zoom: return diagnostic if CQL sortby conv fails.  Diagnostic Bib-1 214
+"Illegal sort relation" is returned. Gets translated to SRU diagnostic
+90: "Unsupported direction value".
+
+zoom: honor database argument nocproxy. This disables creating a content
+connector session (for non-CF targets, usually) and also relays nocproxy
+as database argument to a CF target.
+
+--- 1.3.19 2011/12/16
+
+zoom: Multiple proxy hosts may be given for proxy rich database 
+parameter. List is comma separated.
+
+http_file: decodes URI paths properly. Does not return 404: not found
+if no prefix are matches. Instead packages are passed on.
+
+--- 1.3.18 2011/12/08
+
+zoom: content-{user,password,proxy} are passed on to content proxy
+system and takes precedence over user,password,proxy .
+
+sru_z3950: SRU extra argument x-log-enable=1 enables a search response
+with a log element including log material from children filters. At
+this point, only the zoom filter uses this log facility. It will
+eventually be used by other filters as well. In particular it might be
+useful for this feature to track all filters in used during a search.
+
+zoom: for x-log-enable=1, APDUs sent by ZOOM C, records as received
+and queries as they are manipulated are logged.
+
+zoom:  extend addinfo with native diagnostics. Diagnostics as received
+by ZOOM C are put into addinfo - including ZOOM, HTTP, SRU and Bib-1
+diagnostics. This indicates both a backend diagnostic and the original
+form.
+
+zoom: default urlRecipe value is ${md-electronic-url}
+
+zoom: target profile setting sruVersion is recognized.
+
+--- 1.3.17 2011/11/17
+
+zoom: log target profile record fetched (AKA Torus).
+
+--- 1.3.16 2011/11/17
+
+Change target profile parameter CfSubDb to CfSubDB.
+
+--- 1.3.15 2011/11/12
+
+zoom: bug fix: realm was relayed to CF targets by mistake (realm only
+makes sense to the zoom module itself).
+
+--- 1.3.14 2011/11/11
+
+zoom: realm may be given as part of database.
+
+--- 1.3.13 2011/10/26
+
+Fix Origin class assigment method; was removd by mistake in 1.3.12.
+
+--- 1.3.12 2011/10/26
+
+zoom: database args x-name=value are passed through to backend database.
+
+SRU filter: user may specify session ID that is passed to log entries
+that follows. It is specified by using x-session-id in the SRU arguments.
+
+zoom: fix null pointer reference which would occur for bad url parameter
+inside "torus" or if url was not given.
+
+--- 1.3.11 2011/09/09
+Fix bug in filter session_shared where sessions would not expire
+properly (session ttl). This in turn could make the target close the
+connection before expected and session_shared would return diagnostic
+2: system temporarily available.
+
+--- 1.3.10 2011/09/09
+
+zoom: new target profile setting: sortmap_field. If sortmap_field is given
+the field is mapped to the value for the sortmap. Values not matching
+sortmap_field are passed verbatim.
+  
+zoom: new target profile setting: sortStrategy. One of: "z3950", "type7",
+"cql", "sru11" or "embed". The "embed" chooses type-7 or CQL sortby
+depending on whether RPN or CQL is actually sent to the target.
+
+--- 1.3.9 2011/08/25
+
+zoom: urlRecipe result is stored in <pz:metadata type="generated-url">.
+
+zoom: For content proxy authentication is stored in param name
+"auth", instead of "authentication".
+
+--- 1.3.8 2011/08/22
+
+zoom: relay schema in SRU mode. Previously recordSchema was not affected
+by elementSet.
+
+--- 1.3.6 2011/08/22
+
+zoom: new target profile element literalTransform that holds XSLT
+(encoded as string) for target. Only if present and non-zero length it is
+considered present. This takes precedence over transform.
+
+Don't rely on trang anymore for Git checkout builds. Configuration in etc
+will only be properly validated if trang is available. From now on only
+relax NC files are bundled with the source dist.
+
+Updates to check for Boost unit test (autoconf stuff). On MacOSX this
+component was not detected when it should be.
+
+--- 1.3.5 2011/08/12
+
+z3950_client: mark non-surrogate diagnostics from backend.
+
+zoom: a proxy may be specified for HTTP fetch of target profiles.
+
+--- 1.3.4 2011/08/01
+
+Honor piggyback. Makes some SRU targets behave that insist on
+maximumRecords > 0, but also increases performance.
+
+--- 1.3.3 2011/08/01
+
+Fix dlpath in installed metaproxy.xml for RPM/Deb package. The
+file was not update for libdir/metaproxy4 in version 1.3.2.
+
+--- 1.3.2 2011/08/01
+
+Change location of Metaproxy modules from libdir/metaproxy to
+libdir/metaproxy4. Directory libdir/metaproxy is also owned by 
+libmetaproxy3 and, thus, should not be used by libmetaproxy4.
+
+New facility to dump APDUs from zoom filter. Enabled by 
+<log apdu="true"/> in the zoom filter configuation.
+
+zoom filter bug fix: SRU diagnostics received from backend was
+not properly converted to Bib-1 diagnostics.
+
+Package metaproxy depends on libmetaproxy4, same version.
+
+--- 1.3.1 2011/07/28
+
+zoom: fix a bug WRT passing authentication parameters to backends.
+
+--- 1.3.0 2011/07/27
+
+Fixes for windows Makefile.
+
+--- 1.2.10 2011/07/25 
+
+Change major version of metaproxy library from 3 to 4. This in turn
+makes new package names for Debian and RPM.
+
+--- 1.2.9 2011/06/27
+
+zoom: enable marc-8 encoding by default for usmarc and opac (in pz2
+transforn mode).
+
+zoom: configurable element sets for record transforms.
+    
+zoom: the target profile based transform is specified by element_transform
+in element torus of of the ZOOM filter configuration ; default value
+is "pz2". Target profile based retrieval is specified by element_raw
+in configuration. It will perform same steps, but omit the XSL transform.
+Default value is "raw".
+
+zoom: element set is omitted if not set in profile. But only if pz2
+transform is in effect. Previously the default element set was to "F".
+
+zoom: For non-CF targets database arguments are parsed. User may supply
+db,user=myuser&password=mypass for non-CF targets. These values override
+what's given the authentication from the target profile.
+
+session_shared: fix reference to freed memory. Would occur if a search
+response had non-surrogate diagnostics.
+
+--- 1.2.8 2011/06/22
+
+Route may be given for each port in frontend_net.
+
+--- 1.2.7 2011/06/22
+
+Documentation for zoom filter. Local target profiles may be specified.
+
+Fixes for zoom filter.
+
+--- 1.2.6 2011/06/15
+
+First version with zoom filter.
+
 --- 1.2.5 2011/05/06
 
 Filter z3950_client may be configured to always send a close APDU
@@ -5,7 +706,7 @@ to backend target when connection/session is closed. This will only
 happen if close APDU is not already sent. Enabled by configuration
 for z3950_client (force_close).
 
-Filter virt_db and multi relays close APDU.
+Filter virt_db and multi: relay close APDU.
 
 Filter virt_db and multi: relay init size parameters 
 preferredMessageSize and maximumRecordSize.