X-Git-Url: http://git.indexdata.com/?a=blobdiff_plain;f=util%2Fregression%2FREADME;fp=util%2Fregression%2FREADME;h=0d7b96bac4735e0b50b4686415d941e78c6dc49e;hb=03ef3621fa8624d56eef1f85274de365ddafed90;hp=0000000000000000000000000000000000000000;hpb=d48bccaed9308a737cdfa55ba555891d0ad85b87;p=cql-java-moved-to-github.git

diff --git a/util/regression/README b/util/regression/README
new file mode 100644
index 0000000..0d7b96b
--- /dev/null
+++ b/util/regression/README
@@ -0,0 +1,92 @@
+
+If you just don't want to think about it
+----------------------------------------
+
+Just use "make" to run regression tests.
+
+
+CQL-Java's regression-testing framework
+---------------------------------------
+
+"queries.raw" is the file of test queries as provided by Rob.
+"mktests" parses the raw file into sections and individual queries
+"sections" is the top-level directory created by that program.
+  "01", "02" etc. represent the sections within the raw file
+    "01/name", "02/name", etc. contain the names of the sections.
+    "01/01.cql", "01/02.cql" etc. are the CQL queries themselves.
+"mkanswers" uses a trusted CQL compiler to generate corresponding XCQL.
+    "01/01.xcql", "01/02.xcql" etc. are the compiled XCQL queries.
+
+Apart from the CQL files, all of the files described to this point are
+included in the distribution, with the "trusted" XCQL output produced
+by my own compiler, and used for regression testing of new versions.
+The CQL files are re-created from "queries.raw" as required.  But
+you're welcome to "make refclean" and rebuild it with mkanswers, to
+contain the trusted compiler output of your choice.
+
+"runtests" compares the output of a nominated CQL compiler with
+existing XCQL files.  Most often, you'll use this to compare the
+results of your own build of CQL-Java with those of my build.  I'll
+use it to test new versions, and people who've written other compilers
+can use it to test their code.  (The code of "runtests" and
+"mkanswers" is worryingly similar: they should probably be the same
+program invoked with different command-line arguments.)
+
+"Makefile" controls the building of all this.  You'll need to edit it
+if you want to use different compilers and suchlike from what's
+written into it, so it may be easier to run the tests by hand -- but
+it's a useful reference for the kinds of commands you might need,
+anyway.  In general, "make" will run the regression tests, creating
+whatever CQL and/or XCQL files it needs; if you do "make refclean"
+first, then the next "make" will rebuild the reference results.
+
+So, for example, if you think Rob Sanderson's parser, CQLParser.py, is
+reliable, and you want to test my parser, CQL-Java's CQLParser class,
+against its results, do this:
+
+	make refclean
+	./mktests queries.raw
+	./mkanswers CQLParser.py
+	./runtests ../../bin/CQLParser ./xmlpp.pl
+
+The second argument to ./runtests is the name of a program to use to
+normalise XML, so that the trusted output and the output being tested
+can be compared for equivalence rather than just for being
+byte-identical.  (If you want to test for byte-identical XCQL, then
+use "cat" as the second argument.)  xmlpp.pl is a fine XML
+pretty-printer from DecisionSoft, found at
+	http://software.decisionsoft.com/tools.html
+
+"showtest" can be used to run a single test showing more details of
+what goes wrong, if anything.  You don't need it as part of the
+regression test, but it's useful when debugging.
+
+Finally, "runcanon" checks that each of the queries when compiled and
+decompiled back to CQL (i.e. canonicalised) remains identical when
+recompiled and redecompiled.
+
+
+Appendix: queries that should fail
+----------------------------------
+
+The following queries are included in Rob's master list, in a final
+section called "Invalid searches [should error]".  They are all
+expected to fail in various ways.  I've taken them out of
+"queries.raw" because it's uninteresting, not to mention rather
+disturbing, to watch compilers fail.  More important, I think, to
+demonstrate correct behaviour for the known-to-work queries.
+
+>
+===
+cat or
+index any 
+index any/wrong term
+a prox/wrong b
+()
+(a
+index any fish)
+(cat any dog or ())
+title = ("illegal parentheses")
+"quoted" any "illegal quotes"
+> illegal="urn:missingQuery"
+"fish" and > illegal="urn:invalidPrefixLocation" "chips"