finished test ICU stand-allone program for benchmarking of ICU tokenization and norma...
authorMarc Cromme <marc@indexdata.dk>
Tue, 22 May 2007 21:20:10 +0000 (21:20 +0000)
committerMarc Cromme <marc@indexdata.dk>
Tue, 22 May 2007 21:20:10 +0000 (21:20 +0000)
commitd060969f7c7f2a41142ae5dfdb945cda973c91ee
treeeea495ae16d5d19c8be1b79fca6d2a163b6e152c
parent4fc03d50d3638f680887d012e6d0586aa8560d8f
finished test ICU stand-allone program for benchmarking of ICU tokenization and normalization. Works quite well, benchmarking on the James English Bible from Project Gutenberg (4,5 MB plain text consisting of 870.000 individual tokens) took 3.5 seconds on a laptop. More testing/benchmarking is needed.
src/Makefile.am
src/icu_chain_test.c