VECTOR SEARCH IN EMBEDDED DATABASES

1 . A computer-implemented method for processing queries based on vectors stored in a database, the computer-implemented method comprising:

storing in the database, a plurality of records;

receiving a plurality of vectors, wherein a vector comprises a sequence of numeric values, each vector associated with a record of the plurality of records;

storing, in a persistent storage, a vector index comprising tuples, wherein each tuple stores a vector and an identifier of a record associated with the vector;

receiving a database query requesting a result set associated with a target record;

identifying a target vector associated with the target record;

accessing the vector index to retrieve a subset of vectors based on a distance from the target vector, wherein the subset of vectors is loaded in memory from the persistent storage;

determining a result set based on the subset of vectors; and

sending the result set.

2 . The computer-implemented method of claim 1 , wherein the vector index is generated by performing steps comprising:

receiving an initial set of vectors;

clustering the initial set of vectors to generate a plurality of clusters of vectors, each cluster associated with a centroid vector, wherein the vector index stores an identifier of a centroid associated with a vector; and

repeatedly performing:

receiving a new vector;

responsive to receiving the new vector, identifying a cluster associated with the new vector based on a vector distance between the new vector and centroid vectors associated with one or more clusters; and

updating the vector index with a mapping of the new vector and the cluster identified.

3 . The computer-implemented method of claim 2 , wherein retrieving a subset of vectors comprises:

identifying a set of centroids associated with the target vector; and

selecting a subset of tuples from the vector index based on the distance of the vector of each tuple, wherein each tuple selected has a centroid matching a centroid from the set of centroids.

4 . The computer-implemented method of claim 1 , wherein vector index is stored in a database table, wherein each record of the database table stores a vector and an identifier of the record associated with the vector.

5 . The computer-implemented method of claim 1 , wherein a record represents a document, wherein the vector is generated from at least a portion of the document.

6 . The computer-implemented method of claim 5 , wherein the vector is generated from one of:

a text portion of the document;

an image attribute of the document;

a video attribute of the document; or

an audio attribute of the document.

7 . The computer-implemented method of claim 1 , wherein the database is an embedded database of an application executing on a memory-constrained device.

8 . A non-transitory computer readable storage medium, storing instructions that when executed by one or more computer processors cause the one or more computer processors to perform steps of a method for processing queries based on vectors stored in a database, the steps comprising:

storing in the database, a plurality of records;

receiving a plurality of vectors, wherein a vector comprises a sequence of numeric values, each vector associated with a record of the plurality of records;

storing, in a persistent storage, a vector index comprising tuples, wherein each tuple stores a vector and an identifier of a record associated with the vector;

receiving a database query requesting a result set associated with a target record;

identifying a target vector associated with the target record;

accessing the vector index to retrieve a subset of vectors based on a distance from the target vector, wherein the subset of vectors is loaded in memory from the persistent storage;

determining a result set based on the subset of vectors; and

sending the result set.

9 . The non-transitory computer readable storage medium of claim 8 , wherein the instructions further cause the one or more computer processors to perform steps for generating the vector index, comprising:

receiving an initial set of vectors;

clustering the initial set of vectors to generate a plurality of clusters of vectors, each cluster associated with a centroid vector, wherein the vector index stores an identifier of a centroid associated with a vector; and

repeatedly performing:

receiving a new vector;

responsive to receiving the new vector, identifying a cluster associated with the new vector based on a vector distance between the new vector and centroid vectors associated with one or more clusters; and

updating the vector index with a mapping of the new vector and the cluster identified.

10 . The non-transitory computer readable storage medium of claim 9 , wherein retrieving a subset of vectors comprises:

identifying a set of centroids associated with the target vector; and

selecting a subset of tuples from the vector index based on the distance of the vector of each tuple, wherein each tuple selected has a centroid matching a centroid from the set of centroids.

11 . The non-transitory computer readable storage medium of claim 8 , wherein vector index is stored in a database table, wherein each record of the database table stores a vector and an identifier of the record associated with the vector.

12 . The non-transitory computer readable storage medium of claim 8 , wherein a record represents a document, wherein the vector is generated from at least a portion of the document.

13 . The non-transitory computer readable storage medium of claim 12 , wherein the vector is generated from one of:

a text portion of the document;

an image attribute of the document;

a video attribute of the document; or

an audio attribute of the document.

14 . The non-transitory computer readable storage medium of claim 8 , wherein the database is an embedded database of an application executing on a memory-constrained device.

15 . A computer system comprising:

one or more computer processors; and

a non-transitory computer readable storage medium, storing instructions that when executed by one or more computer processors cause the one or more computer processors to perform steps of a method for processing queries based on vectors stored in a database, the steps comprising:

storing in the database, a plurality of records;

receiving a plurality of vectors, wherein a vector comprises a sequence of numeric values, each vector associated with a record of the plurality of records;

storing, in a persistent storage, a vector index comprising tuples, wherein each tuple stores a vector and an identifier of a record associated with the vector;

receiving a database query requesting a result set associated with a target record;

identifying a target vector associated with the target record;

accessing the vector index to retrieve a subset of vectors based on a distance from the target vector, wherein the subset of vectors is loaded in memory from the persistent storage;

determining a result set based on the subset of vectors; and

sending the result set.

16 . The computer system of claim 15 ,

wherein the instructions further cause the one or more computer processors to perform steps for generating the vector index, comprising:

receiving an initial set of vectors;

clustering the initial set of vectors to generate a plurality of clusters of vectors, each cluster associated with a centroid vector, wherein the vector index stores an identifier of a centroid associated with a vector; and

repeatedly performing:

receiving a new vector;

responsive to receiving the new vector, identifying a cluster associated with the new vector based on a vector distance between the new vector and centroid vectors associated with one or more clusters; and

updating the vector index with a mapping of the new vector and the cluster identified.

17 . The computer system of claim 16 , wherein retrieving a subset of vectors comprises:

identifying a set of centroids associated with the target vector; and

selecting a subset of tuples from the vector index based on the distance of the vector of each tuple, wherein each tuple selected has a centroid matching a centroid from the set of centroids.

18 . The computer system of claim 15 , wherein vector index is stored in a database table, wherein each record of the database table stores a vector and an identifier of the record associated with the vector.

19 . The computer system of claim 15 , wherein a record represents a document, wherein the vector is generated from at least a portion of the document.

20 . The computer system of claim 19 , wherein the vector is generated from one of:

a text portion of the document;

an image attribute of the document;

a video attribute of the document; or

an audio attribute of the document.