Vector database smart auto-scaling

1 . A method for processing requests from edge devices, the method comprising:

identifying vectors in a vector database;

indexing the vectors in the vector database to identify clusters;

partitioning, based on the clusters, the vector database to obtain a plurality of partitions;

obtaining a centroid for each partition of the plurality of partitions;

placing each partition of the plurality of partitions on a separate database;

generating read replicas for each partition of the plurality of partitions, wherein each partition of the plurality of partitions is associated with a plurality of read replicas, wherein each read replica comprises a full read-only copy of its associated partition;

receiving, following the generating, a read request from an edge device, wherein the read request comprises a search vector;

identifying partitions of the plurality of partitions based on a comparison of centroids associated with each partition of the plurality of partitions and the search vector to obtain similar partitions;

identifying, for each of the similar partitions, a read replica of the plurality of read replicas with a lowest load to obtain lowest load read replicas;

identifying vectors in each of the lowest load read replicas that are above a similarity threshold when compared to the search vector; and

sending the vectors to the edge device.

2 . The method of claim 1 , further comprising:

receiving outputs from a model using the vectors as inputs to the model; and

presenting the outputs to a user via a graphical user interface.

3 . The method of claim 2 , wherein the model comprises a large language model.

4 . The method of claim 1 , wherein load refers to volume of read request traffic on a respective read replica.

5 . The method of claim 1 , wherein identifying clusters comprises:

comparing at least one feature of each vector to the at least one feature of other vectors in the vector database.

6 . The method of claim 1 , further comprising:

monitoring load on each partition of the plurality of partitions;

making a determination, based upon the monitoring, that a number of read replicas associated with a partition of the plurality of partitions is above a threshold; and

removing, based upon the determination, at least one of the number of read replicas from respective partitions.

7 . A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer to perform a method for processing requests from edge devices, the method comprising:

identifying vectors in a vector database;

indexing the vectors in the vector database to identify clusters;

partitioning, based on the clusters, the vector database to obtain a plurality of partitions;

obtaining a centroid for each partition of the plurality of partitions;

placing each partition of the plurality of partitions on a separate database;

generating read replicas for each partition of the plurality of partitions, wherein each partition of the plurality of partitions is associated with a plurality of read replicas, wherein each read replica comprises a full read-only copy of its associated partition;

receiving, following the generating, a read request from an edge device, wherein the read request comprises a search vector;

identifying partitions of the plurality of partitions based on a comparison of centroids associated with each partition of the plurality of partitions and the search vector to obtain similar partitions;

identifying, for each of the similar partitions, a read replica of the plurality of read replicas with a lowest load to obtain lowest load read replicas;

identifying vectors in each of the lowest load read replicas that are above a similarity threshold when compared to the search vector; and

sending the vectors to the edge device.

8 . The non-transitory CRM of claim 7 , further comprising:

receiving outputs from a model using the vectors as inputs to the model; and

presenting the outputs to a user via a graphical user interface.

9 . The non-transitory CRM of claim 8 , wherein the model comprises a large language model.

10 . The non-transitory CRM of claim 7 , wherein load refers to volume of read request traffic on a respective read replica.

11 . The non-transitory CRM of claim 7 , wherein identifying clusters comprises:

comparing at least one feature of each vector to the at least one feature of other vectors in the vector database.

12 . The non-transitory CRM of claim 7 , further comprising:

monitoring load on each partition of the plurality of partitions;

making a determination, based upon the monitoring, that a number of read replicas associated with a partition of the plurality of partitions is above a threshold; and

removing, based upon the determination, at least one of the number of read replicas from respective partitions.

13 . A system for processing requests from edge devices, the system comprising:

persistent storage; and

a computing device, comprising a processor and memory, programmed to:

identify vectors in a vector database;

index the vectors in the vector database to identify clusters;

partition, based on the clusters, the vector database to obtain a plurality of partitions;

obtain a centroid for each partition of the plurality of partitions;

place each partition of the plurality of partitions on a separate database;

generate read replicas for each partition of the plurality of partitions, wherein each partition of the plurality of partitions is associated with a plurality of read replicas, wherein each read replica comprises a full read-only copy of its associated partition;

receive a read request from an edge device, wherein the read request comprises a search vector;

identify partitions of a plurality of partitions based on a comparison of centroids associated with each partition of the plurality of partitions and the search vector to obtain similar partitions, wherein each partition of the similar partitions is associated with a plurality of read replicas;

identify, for each of the similar partitions, a read replica of the plurality of read replicas with a lowest load to obtain lowest load read replicas;

identify vectors in each of the lowest load read replicas that are above a similarity threshold when compared to the search vector; and

send the vectors to the edge device.

14 . The system of claim 13 , wherein the computing device is further programmed to:

receive outputs from a model using the vectors as inputs to the model; and

present the outputs to a user via a graphical user interface.

15 . The system of claim 14 , wherein the model comprises a large language model.

16 . The system of claim 13 , wherein load refers to volume of read request traffic on a respective read replica.

17 . The system of claim 13 , wherein the computing device is further programmed to:

monitor load on each partition of the plurality of partitions;

make a determination, based upon the monitoring, that a number of read replicas associated with a partition of the plurality of partitions is above a threshold; and

remove, based upon the determination, at least one of the number of read replicas from respective partitions.