X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=doc%2Fbook.xml;h=e3217281c81ca1510adc1ea8a518fb6c93049788;hb=6074cd61a0cb2f2879453dac8b5febf7de2c8df5;hp=4ec781e9e5bc1e477dd9236125f269058d94fc11;hpb=8f48376798d4b43d962726ef68f547cbd471d670;p=pazpar2-moved-to-github.git diff --git a/doc/book.xml b/doc/book.xml index 4ec781e..e321728 100644 --- a/doc/book.xml +++ b/doc/book.xml @@ -6,372 +6,999 @@ %local; %entities; - - %common; + + %idcommon; ]> - - - Pazpar2 - User's Guide and Reference - - SebastianHammer - - - ©right-year; - Index Data - - - - Pazpar2 is a high-performance, user interface-independent, data - model-independent metasearching - middleware featuring merging, relevance ranking, record sorting, - and faceted results. - - - This document is a guide and reference to Pazpar version &version;. - - - - - - - - - - - - - - - - Introduction - - Pazpar2 is a stand-alone metasearch client with a webservice API, designed - to be used either from a browser-based client (JavaScript, Flash, Java, - etc.), from from server-side code, or any combination of the two. - Pazpar2 is a highly optimized client designed to - search many resources in parallel. It implements record merging, - relevance-ranking and sorting by arbitrary data content, and facet - analysis for browsing purposes. It is designed to be data model - independent, and is capable of working with MARC, DublinCore, or any - other XML-structured response format -- XSLT is used to normalize and extract - data from retrieval records for display and analysis. It can be used - against any server which supports the Z39.50 protocol. Proprietary - backend modules can be used to support a large number of other protocols - (please contact Index Data for further information about this). - - - Additional functionality such as - user management, attractive displays are expected to be implemented by - applications that use pazpar2. Pazpar2 is user interface independent. - Its functionality is exposed through a simple REST-style webservice API, - designed to be simple to use from an Ajax-enbled browser, Flash - animation, Java applet, etc., or from a higher-level server-side language - like PHP or Java. Because session information can be shared between - browser-based logic and your server-side scripting, there is tremendous - flexibility in how you implement your business logic on top of pazpar2. - - - Once you launch a search in pazpar2, the operation continues behind the - scenes. Pazpar2 connects to servers, carries out searches, and - retrieves, deduplicates, and stores results internally. Your application - code may periodically inquire about the status of an ongoing operation, - and ask to see records or other result set facets. Result become - available immediately, and it is easy to build end-user interfaces which - feel extremely responsive, even when searching more than 100 servers - concurrently. - - - Pazpar2 is designed to be highly configurable. Incoming records are - normalized to XML/UTF-8, and then further normalized using XSLT to a - simple internal representation that is suitable for analysis. By - providing XSLT stylesheets for different kinds of result records, you - can tune pazpar2 to work against different kinds of information - retrieval servers. Finally, metadata is extracted, in a configurable - way, from this internal record, to support display, merging, ranking, - result set facets, and sorting. Pazpar2 is not bound to a specific model - of metadata, such as DublinCore or MARC -- by providing the right - configuration, it can work with a number of different kinds of data in - support of many different applications. - - - Pazpar2 is designed to be efficient and scalable. You can set it up to - search several hundred targets in parallel, or you can use it to support - hundreds of concurrent users. It is implemented with the same attention - to performance and economy that we use in our indexing engines, so that - you can focus on building your application, without worrying about the - details of metasearch logic. You can devote all of your attention to - usability and let pazpar2 do what it does best -- metasearch. + + Pazpar2 - User's Guide and Reference + + SebastianHammer + + + AdamDickmeiss + + + MarcCromme + + + JakubSkoczen + + &version; + + ©right-year; + Index Data + + + + Pazpar2 is a high-performance, user interface-independent, data + model-independent metasearching + middle-ware featuring merging, relevance ranking, record sorting, + and faceted results. + + + This document is a guide and reference to Pazpar2 version &version;. + + + + + + + + + + + + + + + + Introduction + + Pazpar2 is a stand-alone metasearch client with a web-service API, designed + to be used either from a browser-based client (JavaScript, Flash, Java, + etc.), from server-side code, or any combination of the two. + Pazpar2 is a highly optimized client designed to + search many resources in parallel. It implements record merging, + relevance-ranking and sorting by arbitrary data content, and facet + analysis for browsing purposes. It is designed to be data model + independent, and is capable of working with MARC, DublinCore, or any + other XML-structured response format + -- XSLT is used to normalize and extract + data from retrieval records for display and analysis. It can be used + against any server which supports the + Z39.50 and SRU/SRW + protocol. Proprietary + backend modules can be used to support a large number of other protocols + (please contact Index Data for further information about this). + + + Additional functionality such as + user management, attractive displays are expected to be implemented by + applications that use Pazpar2. Pazpar2 is user interface independent. + Its functionality is exposed through a simple REST-style web-service API, + designed to be simple to use from an Ajax-enabled browser, Flash + animation, Java applet, etc., or from a higher-level server-side language + like PHP or Java. Because session information can be shared between + browser-based logic and your server-side scripting, there is tremendous + flexibility in how you implement your business logic on top of Pazpar2. + + + Once you launch a search in Pazpar2, the operation continues behind the + scenes. Pazpar2 connects to servers, carries out searches, and + retrieves, deduplicates, and stores results internally. Your application + code may periodically inquire about the status of an ongoing operation, + and ask to see records or other result set facets. Result becomes + available immediately, and it is easy to build end-user interfaces which + feel extremely responsive, even when searching more than 100 servers + concurrently. + + + Pazpar2 is designed to be highly configurable. Incoming records are + normalized to XML/UTF-8, and then further normalized using XSLT to a + simple internal representation that is suitable for analysis. By + providing XSLT stylesheets for different kinds of result records, you + can tune Pazpar2 to work against different kinds of information + retrieval servers. Finally, metadata is extracted, in a configurable + way, from this internal record, to support display, merging, ranking, + result set facets, and sorting. Pazpar2 is not bound to a specific model + of metadata, such as DublinCore or MARC -- by providing the right + configuration, it can work with a number of different kinds of data in + support of many different applications. + + + Pazpar2 is designed to be efficient and scalable. You can set it up to + search several hundred targets in parallel, or you can use it to support + hundreds of concurrent users. It is implemented with the same attention + to performance and economy that we use in our indexing engines, so that + you can focus on building your application, without worrying about the + details of metasearch logic. You can devote all of your attention to + usability and let Pazpar2 do what it does best -- metasearch. + + + If you wish to connect to commercial or other databases which do not + support open standards, please contact Index Data. We have a licensing + agreement with a third party vendor which will enable Pazpar2 to access + thousands of online databases, in addition to the vast number of catalogs + and online services that support the Z39.50/SRU/SRW protocols. + + + Pazpar2 is our attempt to re-think the traditional paradigms for + implementing and deploying metasearch logic, with an uncompromising + approach to performance, and attempting to make maximum use of the + capabilities of modern browsers. The demo user interface that + accompanies the distribution is but one example. If you think of new + ways of using Pazpar2, we hope you'll share them with us, and if we + can provide assistance with regards to training, design, programming, + integration with different backends, hosting, or support, please don't + hesitate to contact us. If you'd like to see functionality in Pazpar2 + that is not there today, please don't hesitate to contact us. It may + already be in our development pipeline, or there might be a + possibility for you to help out by sponsoring development time or + code. Either way, get in touch and we will give you straight answers. + + + Enjoy! + + + Pazpar2 is covered by the GNU license version 2. + See for further information. + + + + + Installation + + The Pazpar2 package is very small. It includes documentation as well + as the Pazpar2 server. The package also includes a simple user + interface test1 which consists of a single HTML page and a single + JavaScript file to illustrate the use of Pazpar2. + + + Pazpar2 depends on the following tools/libraries: + + YAZ + + + The popular Z39.50 toolkit for the C language. + YAZ must be compiled with Libxml2/Libxslt support. + + + + International + Components for Unicode (ICU) + + + ICU provides Unicode support for non-English languages with + character sets outside the range of 7bit ASCII, like + Greek, Russian, German and French. Pazpar2 uses the ICU + Unicode character conversions, Unicode normalization, case + folding and other fundamental operations needed in + tokenization, normalization and ranking of records. + + + Compiling, linking, and usage of the ICU libraries is optional, + but strongly recommended for usage in an international + environment. + + + + + + + In order to compile Pazpar2, a C compiler which supports C99 or later + is required. + + +
+ Installation on Unix (from Source) + + The latest source code for Pazpar2 is available from + . + Only few systems have none of the required + tools binary packages. + If, for example, Libxml2/libXSLT libraries + are already installed as development packages use these. + + + + Ensure that the development libraries + header files are + available on your system before compiling Pazpar2. For installation + of YAZ, refer to the YAZ installation chapter. + + + gunzip -c pazpar2-version.tar.gz|tar xf - + cd pazpar2-version + ./configure + make + su + make install + + + The make install will install manpages as well as the + Pazpar2 server, pazpar2, + in PREFIX/sbin. + By default, PREFIX is /usr/local/ . This can be + changed with configure option . + +
+ +
+ Installation on Windows (from Source) + + Pazpar2 can be built for Windows using + Microsoft Visual Studio. + The support files for building YAZ on Windows are located in the + win directory. The compilation is performed + using the win/makefile which is to be + processed by the NMAKE utility part of Visual Studio. - If you wish to connect to commercial or other databases which do not - support open standards, please contact Index Data. We have a licensing - agreement with a third party vendor which will enable pazpar2 to access - thousands of online databases, in addition the vast number of catalogs - and online services that support the Z39.50 protocol. + Ensure that the development libraries + header files are + available on your system before compiling Pazpar2. For installation + of YAZ, refer to the YAZ installation chapter. + It is easiest if YAZ and Pazpar2 are unpacked in the same + directory (side-by-side). - Pazpar2 is our attempt to re-think the traditional paradigms for - implementing and deploying metasearch logic, with an uncompromising - approach to performance, and attempting to make maximum use of the - capabilities of modern browsers. The demo user interface that - accompanies the distribution is but one example. If you think of new - ways of using pazpar2, we hope you'll share them with us, and if we - can provide assistance with regards to training, design, programming, - integration with different backends, hosting, or support, please don't - hesitate to contact us. If you'd like to see functionality in pazpar2 - that is not there today, please don't hesitate to contact us. It may - already be in our development pipeline, or there might be a - possibility for you to help out by sponsoring development time or - code. Either way, get in touch and we will give you straight answers. + The compilation is tuned by editing the makefile of Pazpar2. + The process is similar to YAZ. Adjust the various directories + YAZ_DIR, ZLIB_DIR, .. - Enjoy! + Compile Pazpar2 by invoking nmake in + the win directory. + The resulting binaries of the build process are located in the + bin of the Pazpar2 source + tree - including the pazpar2.exe and necessary DLLs. - - + + The Windows version of Pazpar2 is a console application. It may + be installed as a Windows Service by adding option + -install for the pazpar2 program. This will + register Pazpar2 as a service and use the other options provided + in the same invocation. For example: + + cd \MyPazpar2\etc + ..\bin\pazpar2 -install -c pazpar2.cfg -l pazpar2.log + + The Pazpar2 service may now be controlled via the Service Control + Panel. It may be unregistered by passing the -remove + option. Example: + + cd \MyPazpar2\etc + ..\bin\pazpar2 -remove + + +
- - Pazpar2 License - To be decided and written. - - - - Installation +
+ Installation of test1 interface - Pazpar2 depends on the following tools/libraries: - - YAZ - - - The popular Z39.50 toolkit for the C language. YAZ must be - compiled with Libxml2/Libxslt support. - - - - + In this section we outline how to install a simple interface that + is part of the Pazpar2 source package. Note that Debian users can + save time by just installing package pazpar2-test1. - In order to compile Pazpar2 an ANSI C compiler is - required. The requirements should be the same as for YAZ. + A web server must be installed and running on the system, such as Apache. -
- Installation on Unix (from Source) - - Here is a quick step-by-step guide on how to compile the - tools that Pazpar2 uses. Only few systems have none of the required - tools binary packages. If, for example, Libxml2/libxslt are already - installed as development packages use these. - - - - Ensure that the development libraries + header files are - available on your system before compiling Pazpar2. For installation - of YAZ, refer to the YAZ installation chapter. - + + Start the Pazpar2 daemon using the 'in-source' binary of the Pazpar2 + daemon. On Unix the process is: - gunzip -c pazpar2-version.tar.gz|tar xf - - cd pazpar2-version - ./configure - make - su - make install + cd etc + cp pazpar2.cfg.dist pazpar2.cfg + ../src/pazpar2 -f pazpar2.cfg -t edu.xml -
+ And on Windows: + + cd etc + copy pazpar2.cfg.dist pazpar2.cfg + ..\bin\pazpar2 -f pazpar2.cfg -t edu.xml + + This will start a Pazpar2 listener on port 8004. It will proxy + HTTP requests to localhost - port 80, which we assume will be the regular + HTTP server on the system. Inspect and modify pazpar2.cfg as needed + if this is to be changed. The -t option specifies the list of targets + to use for searches. + + + Make a new console and move to the other stuff. + For more information about pazpar2 options refer to the manpage. + -
- Installation on Debian GNU/Linux - - All dependencies for Pazpar2 are available as - Debian - packages for the sarge (stable in 2005) and etch (testing in 2005) - distributions. - - - The procedures for Debian based systems, such as - Ubuntu is probably similar - + + The test1 UI is located in www/test1. Ensure this + directory is available to the web server by either copying + test1 to the document root, create a symlink or + use Apache's Alias directive. + + + + The interface test1 interface should now be available on port 8004. + + + If you don't see the test1 interface. See if test1 is really available + on the same URL but on port 80. If it's not, the Apache configuration + (or other) is not correct. + + + In order to use Apache as frontend for the interface on port 80 + for public access etc., refer to + . + +
+ +
+ Installation on Debian GNU/Linux + + Index Data provides Debian packages for Pazpar2. These are prepared + for Debian versions Etch and Lenny (as of 2007). + These packages are available at + . + +
+ +
+ Apache 2 Proxy + + Apache 2 has a + + proxy module + which allows Pazpar2 to become a backend to an Apache 2 + based web service. The Apache 2 proxy must operate in the + Reverse Proxy mode. + + + + On a Debian based Apache 2 system, the relevant modules can + be enabled with: - apt-get install libyaz-dev + sudo a2enmod proxy_http + + + + Traditionally Pazpar2 interprets URL paths with suffix + /search.pz2. + The + ProxyPass directive of Apache must be used to map a URL path + the the Pazpar2 server (listening port). + + + - With these packages installed, the usual configure + make - procedure can be used for Pazpar2 as outlined in - . + The ProxyPass directive takes a prefix rather than + a suffix as URL path. It is important that the Java Script code + uses the prefix given for it. -
- + - - Using pazpar2 + + Apache 2 proxy configuration - This chapter provides a general introduction to the use and deployment of pazpar2. + If Pazpar2 is running on port 8004 and the portal is using + search.pz2 inside portal in directory + /myportal/ we could use the following + Apache 2 configuration: + + + ProxyRequests Off + + + AddDefaultCharset off + Order deny,allow + Allow from all + + + ProxyPass /myportal/search.pz2 http://localhost:8004/search.pz2 + ProxyVia Off + + ]]> + +
-
- Pazpar2 and your systems architecture - - Pazpar2 is designed to provide asynchronous, behind-the-scenes - metasearching functionality to your application, exposing this - functionality using a simple webservice API that can be accessed - from any number of development environments. In particular, it is - possible to combine pazpar2 either with your server-side dynamic - website scripting, with scripting or code running in the browser, or - with any combination of the two. Pazpar2 is an excellent tool for - building advanced, Ajax-based user interfaces for metasearch - functionality, but it isn't a requirement -- you can choose to use - pazpar2 entirely as a backend to your regular server-side scripting. - When you do use pazpar2 in conjunction - with browser scripting (JavaScript/Ajax, Flash, applets, etc.), there are - special considerations. - + - - Pazpar2 implements a simple but efficient HTTP server, and it is - designed to interact directly with scripting running in the browser - for the best possible performance, and to limit overhead when - several browser clients generate numerous webservice requests. - However, it is still desirable to use a conventional webserver, - such as Apache, to serve up graphics, HTML documents, and - server-side scripting. Because the security sandbox environment of - most browser-side programming environments only allows communication - with the server from which the enclosing HTML page or object - originated, pazpar2 is designed so that it can act as a transparent - proxy in front of an existing webserver (see for details). In this mode, all regular - HTTP requests are transparently passed through to your webserver, - while pazpar2 only intercepts search-related webservice requests. - + + Using Pazpar2 + + This chapter provides a general introduction to the use and + deployment of Pazpar2. + - - If you want to expose your combined service on port 80, you can - either run your regular webserver on a different port, a different - server, or a different IP address associated with the same server. - +
+ Pazpar2 and your systems architecture + + Pazpar2 is designed to provide asynchronous, behind-the-scenes + metasearching functionality to your application, exposing this + functionality using a simple webservice API that can be accessed + from any number of development environments. In particular, it is + possible to combine Pazpar2 either with your server-side dynamic + website scripting, with scripting or code running in the browser, or + with any combination of the two. Pazpar2 is an excellent tool for + building advanced, Ajax-based user interfaces for metasearch + functionality, but it isn't a requirement -- you can choose to use + Pazpar2 entirely as a backend to your regular server-side scripting. + When you do use Pazpar2 in conjunction + with browser scripting (JavaScript/Ajax, Flash, applets, + etc.), there are special considerations. + - - Sometimes, it may be necessary to implement functionality on your - regular webserver that makes use of search results, for example to - implement data import functionality, emailing results, history - lists, personal citation lists, interlibrary loan functionality - ,etc. Fortunately, it is simple to exchange information between - pazpar2, your browser scripting, and backend server-side scripting. - You can send a session ID and possibly a record ID from your browser - code to your server code, and from there use pazpar2s webservice API - to access result sets or individual records. You could even 'hide' - all of pazpar2s functionality between your own API implemented on - the server-side, and access that from the browser or elsewhere. The - possibilities are just about endless. - -
+ + Pazpar2 implements a simple but efficient HTTP server, and it is + designed to interact directly with scripting running in the browser + for the best possible performance, and to limit overhead when + several browser clients generate numerous webservice requests. + However, it is still desirable to use a conventional webserver, + such as Apache, to serve up graphics, HTML documents, and + server-side scripting. Because the security sandbox environment of + most browser-side programming environments only allows communication + with the server from which the enclosing HTML page or object + originated, Pazpar2 is designed so that it can act as a transparent + proxy in front of an existing webserver (see for details). + In this mode, all regular + HTTP requests are transparently passed through to your webserver, + while Pazpar2 only intercepts search-related webservice requests. + -
- Your data model - - Pazpar2 does not have a preconceived model of what makes up a data - model. There are no assumption that records have specific fields or - that they are organized in any particular way. The only assumption - is that data comes packaged in a form that the software can work - with (presently, that means XML or MARC), and that you can provide - the necessary information to massage it into pazpar2's internal - record abstraction. - + + If you want to expose your combined service on port 80, you can + either run your regular webserver on a different port, a different + server, or a different IP address associated with the same server. + - - Handling retrieval records in pazpar2 is a two-step process. First, - you decide which data elements of the source record you are - interested in, and you specify any desired massaging or combining of - elements using an XSLT stylesheet (MARC records are automatically - normalized to MARCXML before this step). If desired, you can run - multiple XSLT stylesheets in series to accomplish this, but the - output of the last one should be a representation of the record in a - schema that pazpar2 understands. - + + Pazpar2 can also work behind + a reverse Proxy. Refer to ) + for more information. + This allows your existing HTTP server to operate on port 80 as usual. + Pazpar2 can be started on another (internal) port. + - - The intermediate, internal representation of the record looks like - this: - + + Sometimes, it may be necessary to implement functionality on your + regular webserver that makes use of search results, for example to + implement data import functionality, emailing results, history + lists, personal citation lists, interlibrary loan functionality, + etc. Fortunately, it is simple to exchange information between + Pazpar2, your browser scripting, and backend server-side scripting. + You can send a session ID and possibly a record ID from your browser + code to your server code, and from there use Pazpar2s webservice API + to access result sets or individual records. You could even 'hide' + all of Pazpar2s functionality between your own API implemented on + the server-side, and access that from the browser or elsewhere. The + possibilities are just about endless. + +
- The Shining +
+ Your data model + + Pazpar2 does not have a preconceived model of what makes up a data + model. There are no assumptions that records have specific fields or + that they are organized in any particular way. The only assumption + is that data comes packaged in a form that the software can work + with (presently, that means XML or MARC), and that you can provide + the necessary information to massage it into Pazpar2's internal + record abstraction. + - King, Stephen + + Handling retrieval records in Pazpar2 is a two-step process. First, + you decide which data elements of the source record you are + interested in, and you specify any desired massaging or combining of + elements using an XSLT stylesheet (MARC records are automatically + normalized to MARCXML before this step). + If desired, you can run multiple XSLT stylesheets in series to accomplish + this, but the output of the last one should be a representation of the + record in a schema that Pazpar2 understands. + - ebook + + The intermediate, internal representation of the record looks like + this: + - - -]]> + The Shining - As you can see, there isn't much to it. There are really only a few - important elements to this file. - + King, Stephen - - Elements should belong to the namespace - http://www.indexdata.com/pazpar2/1.0. If the root node contains the - attribute 'mergekey', then every record that generates the same - merge key (normalized for case differences, white space, and - truncation) will be joined into a cluster. In other words, you - decide how records are merged. If you don't include a merge key, - records are never merged. The 'metadata' elements provide the meat - of the elements -- the content. the 'type' attribute is used to - match each element against processing rules that determine what - happens to the data element next. - + ebook - - The next processing step is the extraction of metadata from the - intermediate representation of the record. This is governed by the - 'metadata' elements in the 'service' section of the configuration - file. See for details. The metadata - in the retrieval record ultimately drives merging, sorting, ranking, - the extraction of browse facets, and display, all configurable. - -
+ + + ]]> -
- Client development - - You can use pazpar2 from any environment that allows you to use - webservices. The initial goal of the software was to support - Ajax-based applications, but there literally are no limits to what - you can do. You can use pazpar2 from Javascript, Flash, Java, etc., - on the browser side, and from any development environment on the - server side, and you can pass session tokens and record IDs freely - around between these environments to build sophisticated applications. - Use your imagination. - + As you can see, there isn't much to it. There are really only a few + important elements to this file. + - - The webservice API of pazpar2 is described in detail in . - + + Elements should belong to the namespace + http://www.indexdata.com/pazpar2/1.0. + If the root node contains the + attribute 'mergekey', then every record that generates the same + merge key (normalized for case differences, white space, and + truncation) will be joined into a cluster. In other words, you + decide how records are merged. If you don't include a merge key, + records are never merged. The 'metadata' elements provide the meat + of the elements -- the content. the 'type' attribute is used to + match each element against processing rules that determine what + happens to the data element next. + - - In brief, you use the 'init' command to create a session, a - temporary workspace which carries information about the current - search. You start a new search using the 'search' command. Once the - search has been started, you can follow its progress using the - 'stat', 'bytarget', 'termlist', or 'show' commands. Detailed records - can be fetched using the 'record' command. - -
- + + The next processing step is the extraction of metadata from the + intermediate representation of the record. This is governed by the + 'metadata' elements in the 'service' section of the configuration + file. See for details. The metadata + in the retrieval record ultimately drives merging, sorting, ranking, + the extraction of browse facets, and display, all configurable. + +
- - Reference - - - The material in this chapter is drawn directly from the individual - manual entries. - - - &manref; - +
+ Client development overview + + You can use Pazpar2 from any environment that allows you to use + webservices. The initial goal of the software was to support + Ajax-based applications, but there literally are no limits to what + you can do. You can use Pazpar2 from Javascript, Flash, Java, etc., + on the browser side, and from any development environment on the + server side, and you can pass session tokens and record IDs freely + around between these environments to build sophisticated applications. + Use your imagination. + + + + The webservice API of Pazpar2 is described in detail in . + + + + In brief, you use the 'init' command to create a session, a + temporary workspace which carries information about the current + search. You start a new search using the 'search' command. Once the + search has been started, you can follow its progress using the + 'stat', 'bytarget', 'termlist', or 'show' commands. Detailed records + can be fetched using the 'record' command. + +
+ + §-ajaxdev; + +
+ Connecting to non-standard resources + + Pazpar2 uses Z39.50 as its switchboard language -- i.e. as far as it + is concerned, all resources speak Z39.50, or its webservices derivatives, + SRU/SRW. It is, however, equipped + to handle a broad range of different server behavior, through + configurable query mapping and record normalization. If you develop + configuration, stylesheets, etc., for a new type of resources, we + encourage you to share your work. But you can also use Pazpar2 to + connect to hundreds of resources that do not support standard + protocols. + + + + For a growing number of resources, Z39.50 is all you need. Over the + last few years, a number of commercial, full-text resources have + implemented Z39.50. These can be used through Pazpar2 with little or + no effort. Resources that use non-standard record formats will + require a bit of XSLT work, but that's all. + + + + But what about resources that don't support Z39.50 at all? Some resources might + support OpenSearch, private, XML/HTTP-based protocols, or something + else entirely. Some databases exist only as web user interfaces and + will require screen-scraping. Still others exist only as static + files, or perhaps as databases supporting the OAI-PMH protocol. + There is hope! Read on. + + + + Index Data continues to advocate the support of open standards. We + work with database vendors to support standards, so you don't have + to worry about programming against non-standard services. We also + provide tools (see SimpleServer) + which make it comparatively easy to build gateways against servers + with non-standard behavior. Again, we encourage you to share any + work you do in this direction. + + + + But the bottom line is that working with non-standard resources in + metasearching is really, really hard. If you want to build a + project with Pazpar2, and you need access to resources with + non-standard interfaces, we can help. We run gateways to more than + 2,000 popular, commercial databases and other resources, + making it simple + to plug them directly into Pazpar2. For a small annual fee per + database, we can help you establish connections to your licensed + resources. Meanwhile, you can help! If you build your own + standards-compliant gateways, host them for others, or share the + code! And tell your vendors that they can save everybody money and + increase the appeal of their resources by supporting standards. + + + + There are those who will ask us why we are using Z39.50 as our + switchboard language rather than a different protocol. Basically, + we believe that Z39.50 is presently the most widely implemented + information retrieval protocol that has the level of functionality + required to support a good metasearching experience (structured + searching, structured, well-defined results). It is also compact and + efficient, and there is a very broad range of tools available to + implement it. + +
+ +
+ Unicode Compliance + + Pazpar2 is Unicode compliant and language and locale aware but relies + on character encoding for the targets to be specified correctly if + the targets themselves are not UTF-8 based (most aren't). + Just a few bad behaving targets can spoil the search experience + considerably if for example Greek, Russian or otherwise non 7-bit ASCII + search terms are entered. In these cases some targets return + records irrelevant to the query, and the result screens will be + cluttered with noise. + + + While noise from misbehaving targets can not be removed, it can + be reduced using truly Unicode based ranking. This is an + option which is available to the system administrator if ICU + support is compiled into Pazpar2, see + for details. + + + In addition, the ICU tokenization and normalization rules must + be defined in the master configuration file described in + . + +
+ + + + + Reference + + + The material in this chapter is drawn directly from the individual + manual entries. + + + &manref; + + + License + +
GPL + + + Pazpar2, + Copyright © ©right-year; Index Data. + + + + Pazpar2 is free software; you can redistribute it and/or modify it under + the terms of the GNU General Public License as published by the Free + Software Foundation; either version 2, or (at your option) any later + version. + + + + Pazpar2 is distributed in the hope that it will be useful, but WITHOUT ANY + WARRANTY; without even the implied warranty of MERCHANTABILITY or + FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + for more details. + + + + You should have received a copy of the GNU General Public License + along with Pazpar2; see the file LICENSE. If not, write to the + Free Software Foundation, 59 Temple Place - Suite 330, Boston, MA + 02111-1307, USA. + + + + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + +
+ +