Artificial intelligence agent output through caching predicted inputs
The present invention provides a system for processing user queries through an artificial intelligence (“AI”) pipeline, utilizing data chunking, question generation, and AI models to deliver contextually relevant responses. The system includes a server that ingests and chunks datasets, generates vector embeddings, and stores the data in one or more vector databases. An execution engine sends the chunked data to an AI model that generates potential questions tailored to different user personas. These questions, along with their corresponding data chunks, are stored in the database for future retrieval. When a user submits a query, the system semantically compares the query to the pre-generated question vectors and retrieves the most relevant question and associated data chunk. The query is then sent to an external AI model for final response generation. The system provides seamless interaction, delivering optimized, context-aware responses to user queries in real-time.
1 . A method for processing user queries based on a dataset associated with an artificial intelligence (“AI”) agent, comprising:
dividing content into chunks based on chunking parameters configured for semantic processing;
generating questions for each of a plurality of generated chunks, comprising:
sending the respective chunk and a system prompt to an AI model to cause the AI model to generate questions to which the chunk would be responsive;
vectorizing the generated questions to create question vectors; and
storing the question vectors in a first vector database, wherein the question vectors are stored with metadata associating the respective question vector to the respective chunk;
receiving a user query, the user query originating from a user device;
determining which chunks are responsive to the user query by executing a vector-based semantic comparison of the user query against the generated questions, comprising:
vectorizing the user query to create a query vector; and
comparing the query vector to the question vectors of the first vector database to identify a question vector that meets a threshold similarity to the query vector;
retrieving the chunk associated with the metadata of the question vector; and
synthesizing a result that comprises the retrieved chunk, wherein the result is transmitted back to the user device.
2 . The method of claim 1 , wherein the chunks are stored in a second database with each chunk having a chunk identifier, and wherein the metadata of the question vectors in the first vector database references the chunk identifiers.
3 . The method of claim 2 , wherein the second database comprises a vector database.
4 . The method of claim 1 , wherein the system prompt causes the AI model to create questions relating to each of a plurality of personas, and the plurality of questions received from the AI model comprises questions relating to each of the plurality of personas.
5 . The method of claim 4 , wherein at least two of the plurality of personas correspond to different user roles.
6 . The method of claim 1 , further comprising:
causing display of multiple questions at the user device that correspond to question vectors meeting the threshold similarity to the query vector; and
receiving a selection of a first question of the multiple questions, wherein the first question corresponds to the question vector,
wherein the selection causes the result to comprise the retrieved chunk.
7 . The method of claim 1 , wherein synthesizing the result comprises:
submitting, to the AI model or a separate AI model, the user query and the retrieved chunk; and
receiving at least a portion of the result from the AI model or the separate AI model.
8 . The method of claim 1 , wherein the threshold similarity comprises a threshold distance or angle measurement to the question vector.
9 . The method of claim 1 , wherein the threshold similarity comprises matching a role of a user against a role specified by the metadata of the question vector.
10 . The method of claim 1 , wherein the user query is received at an endpoint associated with the AI agent, and wherein executing the AI agent comprises sending the user query to the AI model.
11 . The method of claim 1 , further comprising caching a portion of the synthesized result in association with the question vector.
12 . The method of claim 11 , further comprising retrieving the cached portion of the synthesized result in an instance in which a second query vector is semantically similar to the question vector.
13 . The method of claim 12 , wherein retrieving the cached portion is contingent on a user credential satisfying an access privilege.
14 . The method of claim 1 , wherein the metadata comprises an agent execution instruction, wherein the AI agent performs an action specified by the agent execution instruction.
15 . The method of claim 14 , wherein the action comprises a semantic search of an additional data source.
16 . The method of claim 14 , wherein the action comprises retrieving search keywords from a specified location.
17 . The method of claim 1 , wherein the metadata comprises text of the respective chunk.
18 . The method of claim 1 , wherein the question vector is associated with an access privilege and wherein access to the retrieved chunk is contingent on a user credential satisfying the access privilege.
19 . A non-transitory, computer-readable medium containing instructions that, when executed by a hardware-based processor, causes the processor to perform stages for processing user queries based on a dataset associated with an artificial intelligence (“AI”) agent, comprising:
dividing content into chunks based on chunking parameters configured for semantic processing;
generating questions for each of a plurality of generated chunks, comprising:
sending the respective chunk and a system prompt to an AI model to cause the AI model to generate questions to which the chunk would be responsive;
vectorizing the generated questions to create question vectors; and
storing the question vectors in a first vector database, wherein the question vectors are stored with metadata associating the respective question vector to the respective chunk;
receiving a user query, the user query originating from a user device;
determining which chunks are responsive to the user query by executing a vector-based semantic comparison of the user query against the generated questions, comprising:
vectorizing the user query to create a query vector; and
comparing the query vector to the question vectors of the first vector database to identify a question vector that meets a threshold similarity to the query vector;
retrieving the chunk associated with the metadata of the question vector; and
synthesizing a result that comprises the retrieved chunk, wherein the result is transmitted back to the user device.
20 . A system for processing user queries based on a dataset associated with an artificial intelligence (“AI”) agent, comprising:
a memory storage comprising a non-transitory, computer-readable medium comprising instructions; and
at least one hardware-based processor that executes the instructions to carry out stages comprising:
dividing content into chunks based on chunking parameters configured for semantic processing;
generating questions for each of a plurality of generated chunks, comprising:
sending the respective chunk and a system prompt to an AI model to cause the AI model to generate questions to which the chunk would be responsive;
vectorizing the generated questions to create question vectors; and
storing the question vectors in a first vector database, wherein the question vectors are stored with metadata associating the respective question vector to the respective chunk;
receiving a user query, the user query originating from a user device;
determining which chunks are responsive to the user query by executing a vector-based semantic comparison of the user query against the generated questions, comprising:
vectorizing the user query to create a query vector; and
comparing the query vector to the question vectors of the first vector database to identify a question vector that meets a threshold similarity to the query vector;
retrieving the chunk associated with the metadata of the question vector; and
synthesizing a result that comprises the retrieved chunk, wherein the result is transmitted back to the user device.