Revision 8f8e122...
Go back to digest for 23rd February 2014Optimization in KDE Base
XapianSearchStore: Increase the max number of terms chosen for small queries
When performing queries with very few characters, they need to be
expanded into every possible auto-completion. So searching for "sea*" =>
adding terms for "sea", "season", etc. This is mostly fine if the prefix
is large enough. When the prefix is quite small like one character the
entire list becomes very large and takes a LOT of memory.
Therefore we only add the top x terms with that prefix in the list based
on their frequencies. This works quite well except when there is an AND
in the query because a term may not have been included in the query
because its frequency is very low, but because of the AND all of high
frequency terms are discarded.
I'm increasing the max number of terms these small queries should be
expanded into. This does not solve the problem but does reduce its
likelyhood. Only Xapian can fix this properly and they are working on a
solution in their 1.3 series.
File Changes
- src/xapian/xapiansearchstore.cpp