Private artificial intelligence (AI) searching on a database using a large language model
A private artificial intelligence (AI) search platform enables generation of model responses using private data while maintaining data privacy within a private system boundary. The system receives a user query via a chat interface and retrieves a responsive private data portion from a vector database through a search. This retrieved context is injected into a prompt along with the user query and transmitted to a large language model (LLM). The LLM, which may be self-hosted on a server within the private system boundary, generates a model response based on the private data portion.
1 . A method comprising:
receiving organizational data from a first computing device;
converting, by an embedding model, the organizational data from a non-vector format to a vector format, including:
activating a number of a plurality of inference processors based on a detected size of the organizational data;
partitioning the organizational data into a plurality of portions; and
converting, by the plurality of inference processors, the plurality of portions to the vector format such that the plurality of portions are converted at least partially in parallel with each other;
storing the organizational data in the vector format in a vector database;
receiving, via an input field on a chat interface of a second computing device, a user query;
converting, by the embedding model, the user query to a vectorized user query;
retrieving, from the vector database, an organizational data portion, from the organizational data, that is responsive to the vectorized user query;
transmitting a prompt to a language model, the prompt including the user query and the organizational data portion retrieved from the vector database;
receiving, from the language model, a model response with textual data that responds to the user query; and
initiating display of the model response in the chat interface,
wherein the embedding model and the language model are stored on at least one server computer within a private system boundary separated from network-based services such that the organizational data and the organizational data portion are not transmitted outside the private system boundary when generating the model response.
2 . The method of claim 1 , wherein the language model is a first language model, the method further comprising:
rendering a list of language models for selection by a user;
receiving a user selection of the first language model from the list of language models;
in response to the user selection of the first language model, obtaining, from a memory device, a first connector of a plurality of connectors, the plurality of connectors including the first connector and a second connector, the first connector configured to enable communication with the first language model, the second connector configured to enable communication with a second language model; and
transmitting, using the first connector, the prompt to the first language model.
3 . The method of claim 2 , wherein the prompt is a first prompt, the method further comprising:
receiving a user selection of the second language model from the list of language models;
in response to the user selection of the second language model, obtaining, from the memory device, the second connector of the plurality of connectors; and
transmitting, using the second connector, a second prompt to the second language model.
4 . The method of claim 1 , further comprising:
obtaining a user identifier associated with a user that submitted the user query;
retrieving, from the vector database, personalization data based on the user identifier, the personalization data including document access control data about at least one access permission or restriction associated with the user for accessing the organizational data; and
retrieving, from the vector database, one or more organizational data portions that satisfy the document access control data.
5 . The method of claim 1 , wherein converting the organizational data from the non-vector format to the vector format includes converting an object in memory to a byte stream that can be stored as a binary file on disk and vectoring the byte stream using the embedding model.
6 . An apparatus comprising:
at least one processor; and
a non-transitory computer-readable medium storing executable instructions that cause the at least one processor to execute operations, the operations comprising:
receiving organizational data from a first computing device;
converting, by an embedding model, the organizational data from a non-vector format to a vector format, including:
activating a number of a plurality of inference processors based on a detected size of the organizational data;
partitioning the organizational data into a plurality of portions; and
converting, by the plurality of inference processors, the plurality of portions to the vector format such that the plurality of portions are converted at least partially in parallel with each other;
storing the organizational data in the vector format in a vector database;
receiving, via an input field on a chat interface of a second computing device, a user query;
converting, by the embedding model, the user query to a vectorized user query;
retrieving, from the vector database, an organizational data portion, from the organizational data, that is responsive to the vectorized user query;
transmitting a prompt to a language model, the prompt including the user query and the organizational data portion retrieved from the vector database;
receiving, from the language model, a model response with textual data that responds to the user query; and
initiating display of the model response in the chat interface,
wherein the embedding model and the language model are stored on at least one server computer within a private system boundary separated from network-based services, such that the organizational data and the organizational data portion are not transmitted outside the private system boundary when generating the model response.
7 . The apparatus of claim 6 , wherein the language model is a first language model, wherein the operations further comprise:
rendering a list of language models for selection by a user;
receiving a user selection of the first language model from the list of language models;
in response to the user selection of the first language model, obtaining, from a memory device, a first connector of a plurality of connectors, the plurality of connectors including the first connector and a second connector, the first connector configured to enable communication with the first language model, the second connector configured to enable communication with a second language model; and
transmitting, using the first connector, the prompt to the first language model.
8 . The apparatus of claim 7 , wherein the prompt is a first prompt, wherein the operations further comprise:
receiving a user selection of the second language model from the list of language models;
in response to the user selection of the second language model, obtaining, from the memory device, the second connector of the plurality of connectors; and
transmitting, using the second connector, a second prompt to the second language model.
9 . The apparatus of claim 6 , wherein the language model is a self- hosted model not publicly accessible.
10 . The apparatus of claim 6 , wherein the operations further comprise:
obtaining a user identifier associated with a user that submitted the user query;
retrieving, from the vector database, personalization data based on the user identifier, the personalization data including document access control data about at least one access permission or restriction associated with the user for accessing the organizational data; and
retrieving, from the vector database, one or more organizational data portions that satisfy the document access control data.
11 . A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations comprising:
receiving organizational data from a first computing device;
converting, by an embedding model, the organizational data from a non-vector format to a vector format, including:
activating a number of a plurality of inference processors based on a detected size of the organizational data;
partitioning the organizational data into a plurality of portions; and
converting, by the plurality of inference processors, the plurality of portions to the vector format such that the plurality of portions are converted at least partially in parallel with each other;
storing the organizational data in the vector format in a vector database;
receiving, via an input field on a chat interface of a second computing device, a user query;
converting, by the embedding model, the user query to a vectorized user query;
retrieving, from the vector database, an organizational data portion, from the organizational data, that is responsive to the user query;
transmitting a prompt to a language model, the prompt including the user query and the organizational data portion;
receiving, from the language model, a model response with textual data that responds to the user query; and
initiating display of the model response in the chat interface,
wherein the embedding model and the language model are stored on at least one server computer within a private system boundary separated from network-based services, such that the organizational data and the organizational data portion are not transmitted outside the private system boundary when generating the model response.
12 . The non-transitory computer-readable medium of claim 11 , further comprising:
obtaining a user identifier associated with a user that submitted the user query;
retrieving, from the vector database, personalization data based on the user identifier, the personalization data including document access control data about at least one access permission or restriction associated with the user for accessing the organizational data; and
retrieving, from the vector database, one or more organizational data portions that satisfy the document access control data.