Generation of context for a question submitted to a generative language machine learning model
A method, computer program product, and computer system. An embedding model is applied to a question to generate a question embedding. The question is to be submitted with context to a trained generative language machine learning model. N documents for inferring the context are a subset of a corpus of C documents that was divided into chunks and associated chunk embeddings. A document tracking database is queried to select M unallowed documents due to lack of access permission. A nearest neighbor (NN) search algorithm is determined to be a K-Nearest Neighbor (KNN) or an Approximate Nearest Neighbor (ANN) search algorithm. An ordered list of ranked chunk embeddings, which is generated by executing the NN search algorithm to match the question embedding to a subset of the chunk embeddings, does not include the chuck embeddings of: the M unallowed documents and the C documents exclusive of the N documents.
1 . A method, the method comprising:
applying an embedding model to a question to generate a question embedding associated with the question, the question having been received from a user, the question configured to be submitted with associated context to a trained generative language machine learning model;
receiving a specification of N documents from which the context may be inferred, the N documents being a subset of a corpus of C documents that was divided into chunks and chunk embeddings respectively associated with the chunks, wherein 1<N≤C, wherein the chunk embeddings of the C documents are stored in a vector database, and wherein U unused documents consist of the C documents exclusive of the N documents;
querying a document tracking database to select, from the N documents which are referenced in the document tracking database, M unallowed documents to which the user does not have access permission, wherein 0≤M<N;
determining whether N is less than a number-of-documents threshold (N th ), and if so then determining that a nearest neighbor (NN) search algorithm is a K-Nearest Neighbor (KNN) search algorithm, and if not then determining that the NN search algorithm is an Approximate Nearest Neighbor (ANN) search algorithm; and
generating an ordered list of ranked chunk embeddings, the generating the ordered list comprising performing a NN search using the NN search algorithm to match the question embedding to a subset of the chunk embeddings of the C documents in the vector database, wherein the generated ordered list of ranked chuck embeddings does not include the chunk embeddings of the U unused documents and the chuck embeddings of the M unallowed documents.
2 . The method of claim 1 , the method further comprising:
determining, from the chunks associated with the ranked chunk embeddings, the context associated with the question;
formatting, from the question and the context, a prompt; and
executing the trained generative language machine learning model using the prompt as input, and receiving, from the trained generative language machine learning model, an answer to the prompt.
3 . The method of claim 1 , wherein N<N th , wherein R documents consist of the N documents exclusive of the M unallowed documents, and wherein the performing the NN search comprises performing the KNN search by applying the KNN search algorithm to perform the KNN search to match the question embedding to the chunk embeddings of the R documents in the vector database.
4 . The method of claim 3 , wherein the method comprises:
querying the document tracking database to identify multiple versions of same documents of the N documents having different release dates, wherein the generated ordered list of ranked chunk embeddings does not include non-current chunk embeddings which encompass chunk embeddings of any non-current release version of any same document, and wherein the non-current chunk embeddings are excluded from the KNN search.
5 . The method of claim 3 , wherein N<C.
6 . The method of claim 3 , wherein N═C.
7 . The method of claim 1 , wherein N≥N th , and wherein the performing the NN search comprises:
performing an ANN search by applying the ANN search algorithm to perform the ANN search to match the question embedding to the chunk embeddings of the C documents in the vector database; and
after the performing the ANN search, removing, from the ordered list of ranked chunk embeddings, the chunk embeddings of the U unused documents and the chunk embeddings of the M unallowed documents.
8 . The method of claim 7 , wherein the method comprises:
querying the document tracking database to identify multiple versions of same documents of the N documents having different release dates; and
after the performing the ANN search and the querying the document tracking database, removing, from the ordered list of ranked chunk embeddings, non-current chunk embeddings which are chunk embeddings of any non-current release version of any same document.
9 . The method of claim 7 , wherein N<C.
10 . The method of claim 7 , wherein N═C.
11 . A computer program product, comprising one or more computer readable storage media storing computer readable program instructions, the program instructions executable by one or more processors of a computer system to cause the computer system to perform operations, the operations comprising:
applying an embedding model to a question to generate a question embedding associated with the question, the question having been received from a user, the question configured to be submitted with associated context to a trained generative language machine learning model;
receiving a specification of N documents from which the context may be inferred, the N documents being a subset of a corpus of C documents that was divided into chunks and chunk embeddings respectively associated with the chunks, wherein 1<N≤C, wherein the chunk embeddings of the C documents are stored in a vector database, and wherein U unused documents consist of the C documents exclusive of the N documents;
querying a document tracking database to select, from the N documents which are referenced in the document tracking database, M unallowed documents to which the user does not have access permission, wherein 0≤M<N;
determining whether N is less than a number-of-documents threshold (N th ), and if so then determining that a nearest neighbor (NN) search algorithm is a K-Nearest Neighbor (KNN) search algorithm, and if not then determining that the NN search algorithm is an Approximate Nearest Neighbor (ANN) search algorithm; and
generating an ordered list of ranked chunk embeddings, the generating the ordered list comprising performing a NN search using the NN search algorithm to match the question embedding to a subset of the chunk embeddings of the C documents in the vector database, wherein the generated ordered list of ranked chuck embeddings does not include the chunk embeddings of the U unused documents and the chuck embeddings of the M unallowed documents.
12 . The computer program product of claim 11 , the operations further comprising:
determining, from the chunks associated with the ranked chunk embeddings, the context associated with the question;
formatting, from the question and the context, a prompt; and
executing the trained generative language machine learning model using the prompt as input, and receiving, from the trained generative language machine learning model, an answer to the prompt.
13 . The computer program product of claim 11 , wherein N<N th , wherein R documents consist of the N documents exclusive of the M unallowed documents, and wherein the performing the NN search comprises performing the KNN search by applying the KNN search algorithm to perform the KNN search to match the question embedding to the chunk embeddings of the R documents in the vector database.
14 . The computer program product of claim 13 , wherein the operations further comprise:
querying the document tracking database to identify multiple versions of same documents of the N documents having different release dates, wherein the generated ordered list of ranked chunk embeddings does not include non-current chunk embeddings which encompass chunk embeddings of any non-current release version of any same document, and wherein the non-current chunk embeddings are excluded from the KNN search.
15 . The computer program product of claim 11 , wherein N≥N th , and wherein the performing the NN search comprises:
performing an ANN search by applying the ANN search algorithm to perform the ANN search to match the question embedding to the chunk embeddings of the C documents in the vector database; and
after the performing the ANN search, removing, from the ordered list of ranked chunk embeddings, the chunk embeddings of the U unused documents and the chunk embeddings of the M unallowed documents.
16 . The computer program product of claim 15 , wherein the operations further comprise:
querying the document tracking database to identify multiple versions of same documents of the N documents having different release dates; and
after the performing the ANN search and the querying the document tracking database, removing, from the ordered list of ranked chunk embeddings, non-current chunk embeddings which are chunk embeddings of any non-current release version of any same document.
17 . A computer system, comprising one or more processors, one or more memories, and one or more computer readable storage media, and computer readable program instructions executable by the one or more processors via the one or more memories to cause the computer system to perform operations, the operations comprising:
applying an embedding model to a question to generate a question embedding associated with the question, the question having been received from a user, the question configured to be submitted with associated context to a trained generative language machine learning model;
receiving a specification of N documents from which the context may be inferred, the N documents being a subset of a corpus of C documents that was divided into chunks and chunk embeddings respectively associated with the chunks, wherein 1<N≤C, wherein the chunk embeddings of the C documents are stored in a vector database, and wherein U unused documents consist of the C documents exclusive of the documents;
querying a document tracking database to select, from the N documents which are referenced in the document tracking database, M unallowed documents to which the user does not have access permission, wherein 0≤M<N;
determining whether N is less than a number-of-documents threshold (N th ), and if so then determining that a nearest neighbor (NN) search algorithm is a K-Nearest Neighbor (KNN) search algorithm, and if not then determining that the NN search algorithm is an Approximate Nearest Neighbor (ANN) search algorithm; and
generating an ordered list of ranked chunk embeddings, the generating the ordered list comprising performing a NN search using the NN search algorithm to match the question embedding to a subset of the chunk embeddings of the C documents in the vector database, wherein the generated ordered list of ranked chuck embeddings does not include the chunk embeddings of the U unused documents and the chuck embeddings of the M unallowed documents.
18 . The computer system of claim 17 , the operations further comprising:
determining, from the chunks associated with the ranked chunk embeddings, the context associated with the question;
formatting, from the question and the context, a prompt; and
executing the trained generative language machine learning model using the prompt as input, and receiving, from the trained generative language machine learning model, an answer to the prompt.
19 . The computer system of claim 17 , wherein N<N th , wherein R documents consist of the N documents exclusive of the M unallowed documents, and wherein the performing the NN search comprises performing the KNN search by applying the KNN search algorithm to perform the KNN search to match the question embedding to the chunk embeddings of the R documents in the vector database.
20 . The computer system of claim 17 , wherein N≥N th , and wherein the performing the NN search comprises:
performing an ANN search by applying the ANN search algorithm to perform the ANN search to match the question embedding to the chunk embeddings of the C documents in the vector database; and
after the performing the ANN search, removing, from the ordered list of ranked chunk embeddings, the chunk embeddings of the U unused documents and the chunk embeddings of the M unallowed documents.