relevance: avoid false hits for whitespace tokens
authorAdam Dickmeiss <adam@indexdata.dk>
Tue, 18 Sep 2012 12:32:29 +0000 (14:32 +0200)
committerAdam Dickmeiss <adam@indexdata.dk>
Tue, 18 Sep 2012 12:32:29 +0000 (14:32 +0200)
commitbb28a0c5f71636328d7f48a1a1fad1e07f9b8cd7
treef55505beeefa208998f3d7270fc7c3eb4222abe8
parent35714623860dd8ba20412c4cfbcf3a1a7ce89af8
relevance: avoid false hits for whitespace tokens

For example, & could be turned into an empty string. And that
would occur nowhere else, giving a high inverse document frequency!
src/relevance.c