Granted Patent B1

US 12,450,217 · App. 18/413,532 · Granted Oct 21, 2025

Inventors: Shaunak Godbole (San Francisco, CA); Shih Ping Chang (San Francisco, CA); Clemens Mewald (Lafayette, CA); Anant Bhardwaj (San Francisco, CA)

Assignee: Instabase, Inc.

G06F16/2237G06F40/40

Abstract

Claims (65)

1. A system configured to control retrieval-augmented generation (RAG) using a set of multiple vector databases storing vector embeddings for multiple sets of electronic files, the system comprising:

one or more hardware processors configured by machine readable instructions to:

convert content within a first set of electronic files into a first set of vector embeddings stored in a first vector database;

convert content within a second set of electronic files into a second set of vector embeddings stored in a second vector database;

convert content within a third set of electronic files into a third set of vector embeddings stored in a third vector database;

receive user input representing a user query, wherein the user query entails a set of tasks, wherein the set of tasks includes at least a first task associated with the first set of electronic files and a second task associated with the second set of electronic files;

generate a first agent query based on the first task and a second agent query based on the second task, wherein generating the first and second agent query is performed by the one or more machine learning models;

perform a first selection of a first subset of the first set of vector embeddings based on the first agent query, wherein performing the first selection includes generating, using an embedding model, a first retrieval vector embedding based on the first agent query, and comparing semantic distance between the vector embeddings in the first set and the first retrieval vector embedding;

provide the first subset of the first set of vector embeddings to one or more machine learning models as context for a first prompt based on the first agent query;

provide the first prompt to the one or more machine learning models;

obtain a first reply from the one or more machine learning models in response to the first prompt;

perform a second selection of a second subset of the second set of vector embeddings based on the second agent query, wherein the second selection includes generating, using the embedding model, a second retrieval vector embedding based on the second agent query and comparing semantic distance between the vector embeddings in the second set and the second retrieval vector embedding;

provide the second subset of the second set of vector embeddings to the one or more machine learning models as context for a second prompt based on the second agent query;

provide the second prompt to the one or more machine learning models;

obtain a second reply from the one or more machine learning models in response to the second prompt;

generate a reply to the user query, wherein the reply is based on the first reply and the second reply; and

effectuate a presentation of the reply to a user, through a user interface.

2. The system of claim 1 , wherein the one or more machine learning models include a large language model (LLM).

3. The system of claim 1 , wherein the first, second, and third vector databases are stored in different locations.

4. The system of claim 1 , wherein the first set of vector embeddings has been generated by and received from an embedding model, wherein the embedding model is external to the system.

5. The system of claim 4 , wherein the first set of vector embeddings represent semantic information in the first set of electronic files numerically through hyperdimensional vectors.

6. The system of claim 1 , wherein the first agent query represents the first task numerically through a hyperdimensional vector.

7. The system of claim 1 , wherein generating the reply includes:

(i) generating a prompt based on the first reply and the second reply;

(ii) providing the prompt to the one or more machine learning models; and

(iii) obtaining the reply from the one or more machine learning models.

8. The system of claim 1 , wherein the set of tasks includes a third task associated with the third set of files, wherein the one or more hardware processors are further configured to:

generate a third agent query based on the third task, using the embedding model;

select a third subset of the third set of vector embeddings based on the third agent query;

provide a third subset of the third set of vector embeddings to the one or more machine learning models as context for a third prompt based on the third agent query; and

obtain a third reply from the one or more machine learning models in response to the third prompt;

wherein the reply to the user query is further based on the third reply.

9. The system of claim 1 , wherein the set of tasks includes a third task to be completed using a particular tool, wherein the first reply and the second reply are used as inputs for the third task, wherein the one or more hardware processors are further configured to:

perform the third task using the particular tool,

wherein the reply is based on output from the particular tool.

10. The method of claim 1 , wherein the set of tasks includes a third task associated with the third set of files, the method further comprising:

generating a third agent query based on the third task, using the embedding model;

selecting a third subset of the third set of vector embeddings based on the third agent query;

providing a third subset of the third set of vector embeddings to the one or more machine learning models as context for a third prompt based on the third agent query; and

obtaining a third reply from the one or more machine learning models in response to the third prompt;

wherein the reply to the user query is further based on the third reply.

11. A method of controlling retrieval-augmented generation (RAG) using a set of multiple vector databases storing vector embeddings for multiple sets of electronic files, the method comprising:

converting (i) content within a first set of electronic files into a first set of vector embeddings stored in a first vector database, (ii) content within a second set of electronic files into a second set of vector embeddings stored in a second vector database, and (iii) content within a third set of electronic files into a second set of vector embeddings stored in a third vector database;

receiving user input representing a user query, wherein the user query entails a set of tasks, wherein the set of tasks includes at least a first task associated with the first set of electronic files and a second task associated with the second set of electronic files;

generating, by the one or more machine learning models, a first agent query based on the first task and a second agent query based on the second task;

performing a first selection of a first subset of the first set of vector embeddings based on the first agent query, wherein performing the first selection includes generating, using an embedding model, a first retrieval vector embedding based on the first agent query, and comparing semantic distance between the vector embeddings in the first set and the first retrieval vector embedding;

providing the first subset of the first set of vector embeddings to one or more machine learning models as context for a first prompt based on the first agent query, and providing the first prompt to the one or more machine learning models;

obtaining a first reply from the one or more machine learning models in response to the first prompt;

performing a second selection of a second subset of the second set of vector embeddings based on the second agent query, wherein performing the second selection includes generating, using the embedding model, a second retrieval vector embedding based on the second agent query, and comparing semantic distance between the vector embeddings in the second set and the second retrieval vector embedding;

providing the second subset of the second set of vector embeddings to the one or more machine learning models as context for a second prompt based on the second agent query, and providing the second prompt to the one or more machine learning models;

obtaining a second reply from the one or more machine learning models in response to the second prompt;

generating a reply to the user query, wherein the reply is based on the first reply and the second reply; and

effectuating a presentation of the reply to a user, through a user interface.

12. The method of claim 11 , wherein the one or more machine learning models include a large language model (LLM).

13. The method of claim 11 , wherein the first, second, and third vector databases are stored in different locations.

14. The method of claim 11 , wherein the first set of vector embeddings has been generated by and received from an embedding model, wherein the embedding model is external to the system.

15. The method of claim 14 , wherein the first set of vector embeddings represent semantic information in the first set of electronic files numerically through hyperdimensional vectors.

16. The method of claim 11 , wherein the first agent query represents the first task numerically through a hyperdimensional vector.

17. The method of claim 11 , wherein generating the reply includes:

(i) generating a prompt based on the first reply and the second reply;

(ii) providing the prompt to the one or more machine learning models; and

(iii) obtaining the reply from the one or more machine learning models.

18. The method of claim 11 , wherein the set of tasks includes a third task to be completed using a particular tool, wherein the first reply and the second reply are used as inputs for the third task, wherein the one or more hardware processors are further configured to:

perform the third task using the particular tool,

wherein the reply is based on output from the particular tool.

Assignments (1)

References Cited (89)

Cited By (1)

US 12,650,993