Vector search method and apparatus

1 . A computer-implemented method comprising:

obtaining a query object;

obtaining, by querying a first index, N vector identifiers corresponding to the query object, wherein the first index is in a memory, wherein N is a positive integer, wherein the N vector identifiers comprise first vector identifiers, and wherein each of the first vector identifiers is not to be recalled;

determining, by querying first index entries that are in a second index and that correspond to the first vector identifiers, second vector identifiers corresponding to the first vector identifiers, wherein each of the second vector identifiers is to be recalled, wherein the second index is in a persistent storage medium, and wherein a first compression ratio of a first vector in the first index is greater than a second compression ratio of a second vector in the second index; and

obtaining based on the second vector identifiers, a query result of the query object.

2 . The method of claim 1 , wherein the second index comprises clustering clusters, and wherein each of the clustering clusters comprises second index entries having a same probability of being accessed in a vector search.

3 . The method of claim 2 , wherein each of the second index entries comprises a third vector and an index pointer corresponding to the third vector, and wherein the index pointer is related to a first identifier of a corresponding clustering cluster and a second identifier of the third vector in the corresponding clustering cluster.

4 . The method of claim 2 , further comprising storing the second index entries in a same storage area in the persistent storage medium.

5 . The method of claim 2 , further comprising:

aggregating a first access request for the persistent storage medium; and

aggregating a second access request for the second index entries in a same clustering cluster into a same access request batch.

6 . The method of claim 1 , wherein the N vector identifiers further comprise a third vector identifier that is to be recalled, and wherein the query result is related to the third vector identifier.

7 . The method of claim 1 , further comprising identifying the N vector identifiers to obtain the first vector identifiers and a third vector identifier, wherein the third vector identifier is to be recalled.

8 . The method of claim 7 , wherein each of the N vector identifiers corresponds to a vector in the first index, an original vector, and a quantized loss vector that is based on a loss between the vector in the first index and the original vector, and wherein identifying the N vector identifiers comprises:

obtaining the first vector identifiers through identification based on a first quantized loss value corresponding to each of the first vector identifiers; and

obtaining the third vector identifier through identification based on a second quantized loss value corresponding to the third vector identifier, wherein the second quantized loss value of the third vector identifier in the first index is less than the first quantized loss value of the first vector identifiers in the first index.

9 . The method of claim 1 , further comprising reading a candidate index entry set in the second index, wherein the candidate index entry set comprises the first index entries.

10 . The method of claim 9 , wherein reading the candidate index entry set comprises reading the candidate index entry set from a level 1 cache space in the memory.

11 . The method of claim 10 , wherein the first index corresponds to a vector identifier having a probability of being a first vector identifier that is greater than a preset probability.

12 . The method of claim 9 , wherein reading the candidate index entry set comprises:

reading from a level 1 cache space in the memory to read the candidate index entry set in the second index;

reading from a level 2 cache space in the memory to read the candidate index entry set in the second index when the candidate index entry set in the second index is not successfully read from the level 1 cache space; and

reading from the persistent storage medium to read the candidate index entry set in the second index when the candidate index entry set in the second index is not successfully read from the level 2 cache space.

13 . The method of claim 12 , wherein the level 2 cache space stores first data read from the persistent storage medium in a previous vector search, and wherein the method further comprises updating the first data with second data read from the persistent storage medium.

14 . An apparatus comprising:

a memory configured to store instructions; and

at least one processor coupled to the memory, wherein when executed by the at least one processor, the instructions cause the apparatus to:

obtain a query object;

obtain, by querying a first index, N vector identifiers corresponding to the query object, wherein the first index is in a memory, wherein N is a positive integer, wherein the N vector identifiers comprise a plurality of first vector identifiers, and wherein each of the first vector identifiers is not to be recalled;

determine, by querying first index entries that are in a second index and that correspond to the first vector identifiers, second vector identifiers corresponding to the first vector identifiers, wherein each of the second vector identifiers is to be recalled, wherein the second index is in a persistent storage medium, and wherein a first compression ratio of a first vector in the first index is greater than a second compression ratio of a second vector in the second index; and

obtain, based on the second vector identifiers, a query result of the query object.

15 . The apparatus of claim 14 , wherein the second index comprises clustering clusters, and wherein each of the clustering clusters comprises second index entries having a same probability of being accessed in a vector search.

16 . The apparatus of claim 15 , wherein each of the second index entries comprises a third vector and an index pointer corresponding to the third vector, and wherein the index pointer is related to a first identifier of a corresponding clustering cluster and a second identifier of the third vector in the corresponding clustering cluster.

17 . The apparatus of claim 15 , wherein when executed by the at least one processor the instructions further cause the apparatus to store the second index entries in a same storage area in the persistent storage medium.

18 . The apparatus of claim 15 , wherein when executed by the at least one processor, the instructions further cause the apparatus to:

aggregate a first access request for the persistent storage medium; and

aggregate a second access request for the second index entries in a same clustering cluster into a same access request batch.

19 . The apparatus of claim 14 , wherein the N vector identifiers further comprise a third vector identifier that is to be recalled, and wherein the query result is related to the third vector identifier.

20 . The apparatus of claim 14 , wherein when executed by the at least one processor, the instructions further cause the apparatus to identify the N vector identifiers to obtain the first vector identifiers and a third vector identifier, and wherein the third vector identifier is to be recalled.