A Brief Index for Proximity Searching

Many pattern recognition tasks can be modeled as proximity searching. Here the common task is to quickly find all the elements close to a given query without sequentially scanning a very large database. A recent shift in the searching paradigm has been established by using permutations instead of distances to predict proximity. Every object in the database record how the set of reference objects (the permutants) is seen, i.e. only the relative positions are used. When a query arrives the relative displacements in the permutants between the query and a particular object is measured. This approach turned out to be the roost efficient and scalable, at the expense of loosing recall in the answers. The permutation of every object is represented with kappa, short integers in practice, producing bulky indexes of 16 kappa n, bits. In this paper we show how to represent the permutation as a binary vector, using just one bit for each permutant (instead of log kappa in the plain representation). The Hamming distance in the binary signature is used then to predict proximity between objects in the database. We tested this approach with many real life metric databases obtaining faster queries with a recall close to the Spearman rho using 16 times less space.