BUILDING A SIMPLE HTTP-TO-Z3950 GATEWAY USING YAZ4J AND TOMCAT
==============================================================

[Yaz4J](http://www.indexdata.com/yaz4j) is a wrapper library over the 
client-specific parts of YAZ, a C-based Z39.50 toolkit, and allows you to use 
the ZOOM API directly from Java. Initial version of Yaz4j has been written
by Rob Styles from [Talis](http://www.talis.com) and the project is now 
developed and maintained at IndexData.
[ZOOM](http://zoom.z3950.org/api/zoom-1.4.html) is a relatively straightforward 
API and with a few lines of code you can write a basic application that can 
establish connection to a Z39.50 server. 
Here we will try to build a very simple HTTP-to-Z3950 gateway using yaz4j and 
the Java Servlet technology.

### COMPILING AND INSTALLING YAZ4J

Yaz4j is still an experimental piece of software and as such is not distributed
via Index Data's public Debian Apt repository and there is no Windows build (yet)
either. While it is possible to use the pre-built Linux binaries, users of 
other OSes will have to compile yaz4j from source. No need to worry (yet) - the 
process of compiling yaz4j is quite simple and we will be up and running in no 
time :).

As a prerequisite, to complete th build process you will need JDK, Maven, Swig
and Yaz (development package) installed on your machine. On Debian/Ubuntu you
can get those easily via apt:

    apt-get install sun-java6-jdk maven2 libyaz4-dev swig


The Yaz4j's source code can be checked-out out from our 
[Git repository](http://git.indexdata.com/?p=yaz4j.git;a=summary), and assuming
you have Git installed on your machine you can do that with:

    git clone git://git.indexdata.com/yaz4j

The compilation of both native and Java source code is controlled by Maven2, 
to build the library, invoke the following commands:

    cd yaz4j
    mvn install

That's it. If the build has completed successfully you end up with two files: 
os-independent jar archive with Java ZOOM API classes 
(yaz4j/any/target/yaz4j-any-VERSION.jar) and os-dependent shared library 
(yaz4j/linux/target/libyaz4j.so or yaz4j/win32/target/yaz4j.dll) that contains 
all necessary JNI "glue" to make the native calls possible from Java. If we were
writing a command line Java application, like any other external Java library, 
yaz4j-any-VERSION.jar would have to be placed on your application classpath 
and the native, shared library would have to be added to your system shared 
library path (LD_LIBRARY_PATH on linux, PATH on Windows) or specified as a 
Java system property (namely the java.library.path) just before your 
application is executed:

    java -cp /path/to/yaz4j-*.jar -Djava.library.path=/path/to/libyaz4j.so MyApp

### SETTING UP THE DEVELOPMENT ENVIRONMENT

Setting up a development/runtime environment for a web (servlet) application is 
a bit more complicated. First, you are not invoking the JVM directly, but the 
servlet container (e.g Tomcat) run-script is doing that for you. At this 
point the shared library (so or dll) has to be placed on the servlet container's
shared libraries load path. Unless your library is deployed to the standard 
system location for shared libs (`/usr/lib` on Linux) or it's location is 
already added to the path, the easiest way to do this in Tomcat is by editing 
(create it if it does not exist) the `CATALINA_HOME/bin/setenv.sh` (setenv.bat 
on Windows) script and putting the following lines in there:

    LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/libyaz4j.so
    export LD_LIBRARY_PATH

on Windows (though no Windows build is yet provided)

     set PATH=%PATH;X:\path\to\yaz4j.dll

That's one way of doing it, another would be to alter the standard set of
arguments passed to the JVM before the Tomcat starts and add 
`-Djava.library.path=/path/to/lib there`. Depending on a situation this might 
be preferable/easier (on Debian/Ubuntu you can specify JVM arguments in the
`/etc/default/tomcat6` file).

With the shared library installed we need to install the pure-Java yaz4j-any*jar
with ZOOM API classes by placing it in Tomcat's `lib` directory 
(CATALINA_HOME/lib). As this library makes the Java System call to load the 
native library into the JVM you cannot simply package it along with your web
application (inside the .war file) - it would try to load the library each time
you deploy the webapp and all consecutive deployments would fail.

### WRITING A SERVLET-BASED GATEWAY

With your servlet environment set up all that is left is to write the actual 
application (peanuts :)). At IndexData we use Maven for managing builds of our 
Java software components but Maven is also a great tool for quickly starting up 
a project. To generate a skeleton for our webapp use the Maven archetype plugin:

    mvn -DarchetypeVersion=1.0.1 -Darchetype.interactive=false \
    -DarchetypeArtifactId=webapp-jee5 -DarchetypeGroupId=org.codehaus.mojo.archetypes \ 
    -Dpackage=com.indexdata.zgate -DgroupId=com.indexdata -DartifactId=zgate \ 
    archetype:generate --batch-mode

This will generate a basic webapp project structure:

<blockcode>

|-- pom.xml
`-- src
|-- main
|   |-- java
|   |   `-- com
|   |       `-- indexdata
|   |           `-- zgate
|   `-- webapp
|       |-- WEB-INF
|       |   `-- web.xml
|       `-- index.jsp
`-- test
    `-- java
        `-- com
            `-- indexdata
                `-- zgate

</blockcode>

Maven has already added basic JEE APIs for web development as the project 
dependencies, we need to do the same for yaz4j, so edit the `pom.xml` and 
add the following lines in the `dependencies` section:

<blockcode type="xml">

<dependency>
  <groupId>org.yaz4j</groupId>
  <artifactId>yaz4j-any</artifactId>
  <version>VERSION</version>
  <scope>provided</scope>
</dependency>

</blockcode>

It's crucial that the scope of this dependency is set to `provided` otherwise
the library would end up packaged in the .war archive and we don't want that.

The implementation of our simple gateway will be contained in a single servlet -
`ZGateServlet` - which we need to place under `src/main/webapp/com/indexdata/zgate.` 
The gateway will work by answering HTTP GET requests and will be controlled 
solely by HTTP parameters, the servlet doGet method is shown below:

<blockcode type="java">

protected void doGet(HttpServletRequest request, HttpServletResponse response)
  throws ServletException, IOException {
    String zurl = request.getParameter("zurl");
    if (zurl == null || zurl.isEmpty()) { 
      response.sendError(400, "Missing parameter 'zurl'"); 
      return; 
    }

    String query = request.getParameter("query");
    if (query == null || query.isEmpty()) { 
      response.sendError(400, "Missing parameter 'query'"); 
      return; 
    }

    String syntax = request.getParameter("syntax");
    if (syntax == null || syntax.isEmpty()) { 
      response.sendError(400, "Missing parameter 'syntax'"); 
      return; 
    }

    int maxrecs=10;
    if (request.getParameter("maxrecs") != null 
      && !request.getParameter("maxrecs").isEmpty()) {
      try {
        maxrecs = Integer.parseInt(request.getParameter("maxrecs"));
      } catch (NumberFormatException nfe) {
        response.sendError(400, "Malformed parameter 'maxrecs'");
        return;
      }
    }

    response.getWriter().println("SEARCH PARAMETERS");
    response.getWriter().println("zurl: " + zurl);
    response.getWriter().println("query: " + query);
    response.getWriter().println("syntax: " + syntax);
    response.getWriter().println("maxrecs: " + maxrecs);
    response.getWriter().println();

    Connection con = new Connection(zurl, 0);
    con.setSyntax(syntax);
    try {
      con.connect();
      ResultSet set = con.search(query, Connection.QueryType.PrefixQuery);
      response.getWriter().println("Showing " + maxrecs + " of "+set.getSize());
      response.getWriter().println();
      for(int i=0; i<set.getSize() && i<maxrecs; i++) {
        Record rec = set.getRecord(i);
        response.getWriter().print(rec.render());
      }
    } catch (ZoomException ze) {
      throw new ServletException(ze);
    } finally {
      con.close();
    }
  }

</blockcode>

With the code in-place we can try to compile the project:

    mvn compile

If all is OK, the next step is to register our servlet and map it to an URL in 
src/main/webapp/WEB-INF/web.xml:

<blockcode type="xml">

<servlet>
  <servlet-name>ZgateServlet</servlet-name>
  <servlet-class>com.indexdata.zgate.ZgateServlet</servlet-class>
</servlet>
<servlet-mapping>
  <servlet-name>ZgateServlet</servlet-name>
  <url-pattern>/zgate</url-pattern>
</servlet-mapping>

</blockcode>

On top of that, we will also  make sure that our servlet is automatically 
triggered when accessing the root path of our application:

<blockcode type="xml">

<welcome-file-list>
  <welcome-file>zgate</welcome-file>
  <welcome-file>index.jsp</welcome-file>
</welcome-file-list>

</blockcode>

Now we are ready to build our webapp:

    mvn package

The resulting .war archive is located under `target/zgate.war`, we can deploy
it on tomcat (e.g by using the `/admin` Tomcat admin  console) and test by 
issuing the following request with your browser or curl 
(assuming Tomcat is running on localhost:8080):

    http://localhost:8080/zgate/?zurl=z3950.loc.gov:7090/voyager&query=@attr%201=7%200253333490&syntax=usmarc

  
That's it! You just build yourself a HTTP-to-Z3950 gateway! Just be careful
with exposing it to the outside world - it's not very secure and could be 
easily exploited. The source code and the gateway's Maven project is available 
in the Yaz4j's Git repo under `examples/zgate`. In the meantime, IndexData 
is working on a Debian/Ubuntu package to make the installation of Yaz4j and 
Tomcat configuration greatly simplified - so stay tuned!. If you are interested 
in Windows support - e.g. Visual Studio based build or an installer - please 
let us know.