Locality-sensitive hashing

In computer science, locality-sensitive hashing (LSH) is an algorithmic technique that hashes similar input items into the same "buckets" with high probability. (The number of buckets is much smaller than the universe of possible input items.) Since similar items end up in the same buckets, this technique can be used for data clustering and nearest neighbor search. It differs from conventional hashing techniques in that hash collisions are maximized, not minimized. Alternatively, the technique can be seen as a way to reduce the dimensionality of high-dimensional data; high-dimensional input items can be reduced to low-dimensional versions while preserving relative distances between items. Hashing-based approximate nearest-neighbor search algorithms generally use one of two main categories of hashing methods: either data-independent methods, such as locality-sensitive hashing (LSH); or data-dependent methods, such as locality-preserving hashing (LPH). Locality-preserving hashing was initially devised as a way to facilitate data pipelining in implementations of massively parallel algorithms that use randomized routing and universal hashing to reduce memory contention and network congestion. An LSH family is defined for a metric space , a threshold , an approximation factor , and probabilities and . This family is a set of functions that map elements of the metric space to buckets . An LSH family must satisfy the following conditions for any two points and any hash function chosen uniformly at random from : if , then (i.e., p and q collide) with probability at least , if , then with probability at most . A family is interesting when . Such a family is called -sensitive. Alternatively it is defined with respect to a universe of items U that have a similarity function . An LSH scheme is a family of hash functions H coupled with a probability distribution D over the functions such that a function chosen according to D satisfies the property that for any .