Norm adjusted proximity graph for fast inner product retrieval
Efficient inner product search is important for many data ranking services, such as recommendation and Information Retrieval. Efficient retrieval via inner product dramatically influences the performance of such data searching and retrieval systems. To resolve deficiencies of prior approaches, embodiments of a new index graph construction approach, referred to generally as Norm Adjusted Proximity Graph (NAPG), for approximate Maximum Inner Product Search (MIPS) are presented. With adjusting factors estimated on sampled data, NAPG embodiments select more meaningful data points to connect with when constructing a graph-based index for inner product search. Extensive experiments verify that the improved graph-based index pushes the state-of-the-art of inner product search forward greatly, in the trade-off between search efficiency and effectiveness.
1. A computer-implemented method comprising:
given a vector set comprising multiple vectors:
obtaining a norm range map for the vector set comprising a plurality of sub-ranges;
obtaining an estimated adjusting factor for each sub-range by performing steps comprising:
selecting a set of samples from the sub-range;
for each sample from the set of samples from the sub-range, retrieving top-n vectors from the vector set that have a largest inner product value or values with the sample, wherein n is an integer greater than or equal to 1;
determining a first average of inner product values, in which the inner product values comprise, for each sample in the set of samples, the inner product values between the sample and each of its corresponding retrieved top-n vectors;
determining a second average of inner product values using inner product values obtained from among the retrieved top-n vectors for a sample from the set of samples; and
obtaining the estimated adjusting factor for the sub-range using a division of the second average by the first average;
constructing an index graph using norm-adjusted inner products, in which a norm-adjusted inner product is obtained by adjusting an inner product using one of the estimated adjusting factors; and
given a query vector, performing a search on the constructed index graph to retrieve one or more relevant vectors.
2. The computer-implemented method of claim 1 wherein the multiple sub-ranges are obtained using steps comprising:
obtaining a global norm range in terms of l 2 -norm values for the vector set;
splitting the global norm range evenly into the multiple sub-ranges; and
drawing samples from the vector set using stratification sampling such that each sub-range obtains a same number or within a threshold of the same number of samples.
3. The computer-implemented method of claim 1 wherein the search is a greedy search.
4. The computer-implemented method of claim 1 wherein constructing the index graph using norm-adjusted inner products comprising steps of:
initializing the index graph; and
for each vector in the vector set,
performing a greedy search to obtain k vertices on the index graph that have the top-k largest inner product values with the vector, wherein k is an integer number larger than 1;
initializing a set of selected vertices for the vector;
obtaining an estimated adjusting factor based on the sub-range that the vector belongs to;
updating the set of selected vertices by adding one or more selected vertices, among the k vertices, that have a norm-adjusted inner product between each added vertex and the vector no less than inner products between vertices in the set of selected vertices; and
updating the index graph by adding one or more edges that connect the vector and the one or more selected vertices until the number of vertices in the set of selected vertices reaches a predetermined threshold.
5. The computer-implemented method of claim 4 wherein the index graph is initialized with an empty graph.
6. The computer-implemented method of claim 4 wherein the norm-adjusted inner product between each added vertex and the vector is a product between the estimated adjusting factor and an inner product between each added vertex and the vector.
7. The computer-implemented method of claim 1 wherein the retrieved one or more relevant vectors are ranked in terms of inner products between each relevant vector and the query vector.
8. A system comprising:
one or more processors; and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
given a vector set comprising multiple vectors:
obtaining a norm range map for the vector set comprising a plurality of sub-ranges;
obtaining an estimated adjusting factor for each sub-range by performing steps comprising:
selecting a set of samples from the sub-range;
for each sample from the set of samples from the sub-range, retrieving top-n vectors from the vector set that have a largest inner product value or values with the sample, wherein n is an integer greater than or equal to 1;
determining a first average of inner product values, in which the inner product values comprise, for each sample in the set of samples, the inner product values between the sample and each of its corresponding retrieved top-n vectors;
determining a second average of inner product values using inner product values obtained from among the retrieved top-n vectors for a sample from the set of samples; and
obtaining the estimated adjusting factor for the sub-range using a division of the second average by the first average;
constructing an index graph using norm-adjusted inner products, in which a norm-adjusted inner product is obtained by adjusting an inner product using one of the estimated adjusting factors; and
given a query vector, performing a search on the constructed index graph to retrieve one or more relevant vectors.
9. The system of claim 8 wherein obtaining the multiple sub-ranges comprises steps of:
obtaining a global norm range in terms of l 2 -norm values for the vector set;
splitting the global norm range evenly into the multiple sub-ranges; and
drawing samples from the vector set using stratification sampling such that each sub-range obtains a same number or within a threshold of the same number of samples.
10. The system of claim 8 wherein n is an integer number larger than 1.
11. The system of claim 8 wherein constructing the index graph using norm-adjusted inner products comprising steps of:
initializing the index graph; and
for each vector in the vector set,
performing a greedy search to obtain k vertices on the index graph that have the top-k largest inner product values with the vector, wherein k is an integer number larger than 1;
initializing a set of selected vertices for the vector;
obtaining an estimated adjusting factor based on the sub-range that the vector belongs to;
updating the set of selected vertices by adding one or more selected vertices, among the k vertices, that have a norm-adjusted inner product between each added vertex and the vector no less than inner products between vertices in the set of selected vertices; and
updating the index graph by adding one or more edges that connect the vector and the one or more selected vertices until the number of vertices in the set of selected vertices C reaches a predetermined threshold.
12. The system of claim 11 wherein the index graph is initialized with an empty graph.
13. The system of claim 11 wherein the norm-adjusted inner product between each added vertex and the vector is a product between the estimated adjusting factor and an inner product between each added vertex and the vector.
14. The system of claim 8 wherein the retrieved one or more relevant vectors are ranked in terms of inner products between each relevant vector and the query vector.
15. A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by at least one processor, causes steps to be performed comprising:
given a vector set comprising multiple vectors:
obtaining a norm range map for the vector set comprising a plurality of sub-ranges;
obtaining an estimated adjusting factor for each sub-range by performing steps comprising:
selecting a set of samples from the sub-range;
for each sample from the set of samples from the sub-range, retrieving top-n vectors from the vector set that have a largest inner product value or values with the sample, wherein n is an integer greater than or equal to 1;
determining a first average of inner product values, in which the inner product values comprise, for each sample in the set of samples, the inner product values between the sample and each of its corresponding retrieved top-n vectors;
determining a second average of inner product values using inner product values obtained from among the retrieved top-n vectors for a sample from the set of samples; and
obtaining the estimated adjusting factor for the sub-range using a division of the second average by the first average;
constructing an index graph using norm-adjusted inner products, in which a norm-adjusted inner product is obtained by adjusting an inner product using one of the estimated adjusting factors; and
given a query vector, performing a search on the constructed index graph to retrieve one or more relevant vectors.
16. The non-transitory computer-readable medium or media of claim 15 wherein the multiple sub-ranges are obtained using steps comprising:
obtaining a global norm range in terms of l 2 -norm values for the vector set;
splitting the global norm range evenly into the multiple sub-ranges; and
drawing samples from the vector set using stratification sampling such that each sub-range obtains a same number or within a threshold of the same number of samples.
17. The non-transitory computer-readable medium or media of claim 15 wherein n is an integer number larger than 1; and
the search is a greedy search.
18. The non-transitory computer-readable medium or media of claim 15 wherein constructing the index graph using norm-adjusted inner products comprising steps of:
initializing the index graph; and
for each vector in the vector set,
performing a greedy search to obtain k vertices on the index graph that have the top-k largest inner product values with the vector, wherein k is an integer number larger than 1;
initializing a set of selected vertices for the vector;
obtaining an estimated adjusting factor based on the sub-range that the vector belongs to;
updating the set of selected vertices by adding one or more selected vertices, among the k vertices, that have a norm-adjusted inner product between each added vertex and the vector no less than inner products between vertices in the set of selected vertices; and
updating the index graph by adding one or more edges that connect the vector and the one or more selected vertices until the number of vertices in the set of selected vertices C reaches a predetermined threshold.
19. The non-transitory computer-readable medium or media of claim 18 wherein the norm-adjusted inner product between each added vertex and the vector is a product between the estimated adjusting factor and an inner product between each added vertex and the vector.
20. The non-transitory computer-readable medium or media of claim 15 wherein the retrieved one or more relevant vectors are ranked in terms of inner products between each relevant vector and the query vector.