Information extraction system for unstructured documents using retrieval augmentation with custom query parameters
A system for extracting a number of data elements from one or more unstructured data sources. The system may separate the text from the tables in a document, such that only the table data may be sent to the large language model (LLM), when the LLM only needs to review the table data. The system may form table chunks and text chunks and index the chunks using a vector embedding. The system may, in response to a request to extract the data elements, retrieve and send the targeted table chunks or text chunks to the LLM to extract the data elements. Retrieval of the chunks may be based on different retrieval parameters including a chunk type designation, a document type designation, a search type designation, and a reach criterion. Hierarchical sets of search criteria may be provided to ensure a minimal number of chunks are retrieved.
1 . A method for document retrieval within retrieval augmented generation, the method comprising:
acquiring, by one or more processors, an extraction prompt to cause a large language model to extract information from provided text;
identifying, by the one or more processors, one or more relevant chunks according to retrieval parameters associated with the extraction prompt, the one or more relevant chunks identified from an index of one or more chunks from one or more documents, the index comprising vector text embeddings of the one or more chunks;
generating, by the one or more processors, distance metrics between the vector text embeddings and a vector text embedding of the extraction prompt;
generating, by the one or more processors, keyword scores between the one or more chunks and a keyword associated with the extraction prompt,
wherein the retrieval parameters comprise a search type designation indicating the one or more relevant chunks are to be searched using a semantic search, a keyword search, or both the semantic search and the keyword search, and weighting parameters for a weighted function of the keyword scores and the distance metrics, and wherein identifying the one or more relevant chunks is based on the weighted function of the keyword scores and the distance metrics; and
storing a response from the large language model to the extraction prompt and the one or more relevant chunks.
2 . The method of claim 1 , further comprising forming, by the one or more processors, one or more table chunks having tabular data of the one or more documents and one or more text chunks having text data of the one or more documents, wherein the retrieval parameters comprise a chunk type designation indicating the one or more relevant chunks are to be retrieved from the one or more table chunks, the one or more text chunks, or both the one or more table chunks and the one or more text chunks.
3 . The method of claim 2 , further comprising:
receiving, by the one or more processors, a response payload that comprises document text of a document of the one or more documents and one or more tables of the document represented using markdown language, wherein the response payload is generated from an optical character recognition tool; and
separating, by the one or more processors using the markdown language, the response payload into a first portion having the one or more tables and a second portion having the document text,
wherein forming the one or more table chunks is based on the first portion and forming the one or more text chunks is based on the second portion.
4 . The method of claim 1 , further comprising generating, by the one or more processors, match scores for the one or more chunks using a text-based search, wherein the retrieval parameters comprise one or more regular expressions to perform the text-based search.
5 . The method of claim 1 , wherein the retrieval parameters comprise a document type designation indicating one or more document types from where the one or more relevant chunks are to originate.
6 . The method of claim 1 , wherein the retrieval parameters comprise a search reach criterion, wherein one or more reached chunks that satisfy the search reach criterion with a relevant chunk of the one or more relevant chunks are provided to the large language model.
7 . The method of claim 1 , wherein the retrieval parameters comprise a hierarchy of sets of the retrieval parameters, wherein identifying the one or more relevant chunks according to the retrieval parameters comprises:
identifying, by the one or more processors, the one or more relevant chunks according to a first set of retrieval parameters of the hierarchy;
determining, by the one or more processors, whether the one or more relevant chunks satisfy a retrieval criterion; and
responsive to determining that the one or more relevant chunks do not satisfy the retrieval criterion, identifying, by the one or more processors, the one or more relevant chunks according to a second set of retrieval parameters of the hierarchy.
8 . A system for document retrieval within retrieval augmented generation, the system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
acquiring an extraction prompt to cause a large language model to extract information from provided text;
identifying one or more relevant chunks according to retrieval parameters associated with the extraction prompt, the one or more relevant chunks identified from an index of one or more chunks of one or more documents, the index comprising vector text embeddings of the one or more chunks;
generating distance metrics between the vector text embeddings and a vector text embedding of the extraction prompt;
generating keyword scores between the one or more chunks and a keyword associated with the extraction prompt,
wherein the retrieval parameters comprise a search type designation indicating the one or more relevant chunks are to be searched using a semantic search, a keyword search, or both the semantic search and the keyword search, and weighting parameters for a weighted function of the keyword scores and the distance metrics, and wherein identifying the one or more relevant chunks is based on the weighted function of the keyword scores and the distance metrics; and
storing a response from the large language model to the extraction prompt and the one or more relevant chunks.
9 . The system of claim 8 , the operations further comprising forming, by the one or more processors, one or more table chunks having tabular data of the one or more documents and one or more text chunks having text data of the one or more documents, wherein the retrieval parameters comprise a chunk type designation indicating the one or more relevant chunks are to be retrieved from the one or more table chunks, the one or more text chunks, or both the one or more table chunks and the one or more text chunks.
10 . The system of claim 9 , the operations further comprising:
receiving a response payload that comprises document text of a document of the one or more documents and one or more tables of the document represented using markdown language, wherein the response payload is generated from an optical character recognition tool; and
separating, using the markdown language, the response payload into a first portion having the one or more tables and a second portion having the document text,
wherein forming the one or more table chunks is based on the first portion and forming the one or more text chunks is based on the second portion.
11 . The system of claim 8 , the operations further comprising generating match scores for the one or more chunks using a text-based search, wherein the retrieval parameters comprise one or more regular expressions to perform the text-based search.
12 . The system of claim 8 , wherein the retrieval parameters comprises a document type designation indicating one or more document types from where the one or more relevant chunks are to originate.
13 . The system of claim 8 , wherein the retrieval parameters comprise a search reach criterion, wherein one or more reached chunks that satisfy the search reach criterion with a relevant chunk of the one or more relevant chunks are provided to the large language model.
14 . The system of claim 8 , wherein the retrieval parameters comprise a hierarchy of sets of the retrieval parameters, wherein identifying the one or more relevant chunks according to the retrieval parameters comprises:
identifying the one or more relevant chunks according to a first set of retrieval parameters of the hierarchy;
determining whether the one or more relevant chunks satisfy a retrieval criterion; and
responsive to determining that the one or more relevant chunks do not satisfy the retrieval criterion, identifying the one or more relevant chunks according to a second set of retrieval parameters of the hierarchy.
15 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
generate an index for one or more chunks of one or more documents, wherein generating the index comprises converting the one or more chunks into vector text embeddings using a text embedding model;
acquire an extraction prompt configured to cause a large language model to extract requested data from retrieved chunks of the one or more chunks;
generate distance metrics between the vector text embeddings and a vector text embedding of the extraction prompt;
generate keyword scores between the one or more chunks and a keyword associated with the extraction prompt,
wherein retrieval parameters associated with the extraction prompt comprise a search type designation indicating the one or more relevant chunks are to be searched using a semantic search, a keyword search, or both the semantic search and the keyword search, and weighting parameters for a weighted function of the keyword scores and the distance metrics, and wherein identifying the one or more relevant chunks is based on the weighted function of the keyword scores and the distance metrics;
identify one or more relevant chunks according to a first set of retrieval parameters associated with the extraction prompt;
determine whether the one or more relevant chunks satisfy a retrieval criterion;
identify the one or more relevant chunks according to a second set of retrieval parameters associated with the extraction prompt responsive to determining that the one or more relevant chunks do not satisfy the retrieval criterion; and
store a response from the large language model to the extraction prompt and the one or more relevant chunks.
16 . The non-transitory computer-readable medium of claim 15 , wherein the second set of retrieval parameters comprise a search reach criterion, wherein one or more reached chunks that satisfy the search reach criterion with a relevant chunk of the one or more relevant chunks are provided to the large language model.