Ingestion and retrieval of dynamic source documents in an automated question answering system
A method including generating an index comprising (a) question-answer pairs generated based at least in part on a set of ingested source documents and (b) respective location metadata associated with each of the question-answer pairs. The respective location metadata can include a source document identifier of a first ingested source document of the set of ingested source documents and one or more location delimiters indicating a location of target text in the first ingested source document. The target text can correspond to a stored answer of a respective one of the question-answer pairs associated with the respective location metadata. The method also can include receiving a user question from a user. The user question can correspond to a question of a first question-answer pair of the question-answer pairs of the index. The method additionally can include determining a first answer to the user question based at least in part on the respective location metadata associated with the first question-answer pair. Other embodiments are disclosed.
1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform:
generating an index comprising (a) question-answer pairs generated based at least in part on a set of ingested source documents and (b) respective location metadata associated with each of the question-answer pairs, the respective location metadata comprising a source document identifier of a first ingested source document of the set of ingested source documents and one or more location delimiters indicating a location of target text within the first ingested source document, the target text corresponding to a stored answer of a first question-answer pair of the question-answer pairs associated with the respective location metadata for the first question-answer pair, and the first ingested source document being dynamic and subject to change, wherein generating the index further comprises:
transforming, using machine learning, the one or more location delimiters of the respective location metadata associated with the first question-answer pair into a numeric vector representation of the one or more location delimiters such that the one or more location delimiters is stored in the respective location metadata as the numeric vector representation of the one or more location delimiters;
receiving a user question from a user, the user question corresponding to a question of the first question-answer pair of the question-answer pairs of the index; and
determining a first answer to the user question from a current version of the first ingested source document based at least in part on the respective location metadata associated with the first question-answer pair, wherein the current version of the first ingested source document may be different from an earlier version of the first ingested source document when the first question-answer pair was generated.
2. The system of claim 1 further comprising:
sending the first answer to the user.
3. The system of claim 1 , wherein determining the first answer to the user question further comprises:
determining the first answer to the user question from the current version of the first ingested source document based at least in part on the source document identifier of the respective location metadata associated with the first question-answer pair.
4. The system of claim 1 , wherein determining the first answer to the user question further comprises:
identifying the first ingested source document based on the source document identifier of the respective location metadata associated with the first question-answer pair; and
retrieving the first answer from the current version of the first ingested source document based at least in part on at least one of the one or more location delimiters of the respective location metadata associated with the first question-answer pair.
5. The system of claim 4 , wherein retrieving the first answer from the current version of the first ingested source document further comprises:
determining the first answer from partial matches based at least in part on (a) at least one of the one or more location delimiters and (b) approximate string matching using the current version of the first ingested source document and the target text from the earlier version of the first ingested source document.
6. The system of claim 4 , wherein the first answer is different from the stored answer of the first question-answer pair that was stored in the index, based on the current version of the first ingested source document being different from the earlier version of the first ingested source document when the first question-answer pair was generated.
7. The system of claim 1 , wherein the one or more location delimiters of the respective location metadata comprise at least one of:
a page number of a page of the first ingested source document containing the target text;
a start percentage of a start of the target text within the first ingested source document;
an end percentage of an end of the target text within the first ingested source document;
a preceding sentence occurring immediately before the target text in the first ingested source document; and
a subsequent sentence occurring immediately after the target text in the first ingested source document.
8. The system of claim 1 , wherein generating the index further comprises, for each question-answer pair of the question-answer pairs:
transforming a respective answer of the each question-answer pair into a respective first numeric vector representation; and
transforming each respective location delimiter of the one or more location delimiters of the respective location metadata associated with the each question-answer pair into one or more second respective numeric vector representations,
wherein:
the respective answer of the each question-answer pair is stored in the index as the respective first numeric vector representation; and
the each respective location delimiter of the one or more location delimiters are stored in the respective location metadata as the one or more second respective numeric vector representations.
9. The system of claim 8 , where determining the first answer to the user question further comprises:
transforming into vector embeddings an ingested source document of the set of ingested source documents that is identified by the source document identifier of the respective location metadata associated with the first question-answer pair;
determining similarity scores for portions of the vector embeddings based on similarities of the vector embeddings to the respective first numeric vector representation and the respective one or more second numeric vector representations;
selecting a highest-ranked portion of the portions of the vector embeddings based on a ranking of the portions of the vector embeddings by the similarity scores; and
determining the first answer based on the highest-ranked portion.
10. The system of claim 9 , wherein the similarity scores are determined using a similarity function, the similarity function comprising cosine similarity or Manhattan distance.
11. The system of claim 9 , wherein the respective first numeric vector representation, the respective one or more second numeric vector representations, and the vector embeddings are generated using a neural network.
12. The system of claim 11 , wherein the respective first numeric vector representation, the respective one or more second numeric vector representations, and the vector embeddings are further transformed using a cryptographic hash function.
13. A method being implemented via execution of computing instructions configured to run at one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:
generating an index comprising (a) question-answer pairs generated based at least in part on a set of ingested source documents and (b) respective location metadata associated with each of the question-answer pairs, the respective location metadata comprising a source document identifier of a first ingested source document of the set of ingested source documents and one or more location delimiters indicating a location of target text within the first ingested source document, the target text corresponding to a stored answer of a first question-answer pair of the question-answer pairs associated with the respective location metadata for the first question-answer pair, and the first ingested source document being dynamic and subject to change, wherein generating the index further comprises:
transforming, using machine learning, the one or more location delimiters of the respective location metadata associated with the first question-answer pair into a numeric vector representation of the one or more location delimiters such that the one or more location delimiters is stored in the respective location metadata as the numeric vector representation of the one or more location delimiters;
receiving a user question from a user, the user question corresponding to a question of the first question-answer pair of the question-answer pairs of the index; and
determining a first answer to the user question from a current version of the first ingested source document based at least in part on the respective location metadata associated with the first question-answer pair, wherein the current version of the first ingested source document may be different from an earlier version of the first ingested source document when the first question-answer pair was generated.
14. The method of claim 13 further comprising:
sending the first answer to the user.
15. The method of claim 13 , wherein determining the first answer to the user question further comprises:
determining the first answer to the user question from the current version of the first ingested source document based at least in part on the source document identifier of the respective location metadata associated with the first question-answer pair.
16. The method of claim 13 , wherein determining the first answer to the user question further comprises:
identifying the first ingested source document based on the source document identifier of the respective location metadata associated with the first question-answer pair; and
retrieving the first answer from the current version of the first ingested source document based at least in part on at least one of the one or more location delimiters of the respective location metadata associated with the first question-answer pair.
17. The method of claim 16 , wherein retrieving the first answer from the current version of the first ingested source document further comprises:
determining the first answer from partial matches based at least in part on (a) at least one of the one or more location delimiters and (b) approximate string matching using the current version of the first ingested source document and the target text from the earlier version of the first ingested source document.
18. The method of claim 16 , wherein the first answer is different from the stored answer of the first question-answer pair that was stored in the index, based on the current version of the first ingested source document being different from the earlier version of the first ingested source document when the first question-answer pair was generated.
19. The method of claim 13 , wherein the one or more location delimiters of the respective location metadata comprise at least one of:
a page number of a page of the first ingested source document containing the target text;
a start percentage of a start of the target text within the first ingested source document;
an end percentage of an end of the target text within the first ingested source document;
a preceding sentence occurring immediately before the target text in the first ingested source document; and
a subsequent sentence occurring immediately after the target text in the first ingested source document.
20. The method of claim 13 , wherein generating the index further comprises, for each question-answer pair of the question-answer pairs:
for each question-answer pair of the question-answer pairs:
transforming a respective answer of the each question-answer pair into a respective first numeric vector representation; and
transforming each respective location delimiter of the one or more location delimiters of the respective location metadata associated with the each question-answer pair into one or more second respective numeric vector representations,
wherein:
the respective answer of the each question-answer pair is stored in the index as the respective first numeric vector representation; and
the each respective location delimiter of the one or more location delimiters are stored in the respective location metadata as the one or more second respective numeric vector representations.
21. The method of claim 20 , where determining the first answer to the user question further comprises:
transforming into vector embeddings an ingested source document of the set of ingested source documents that is identified by the source document identifier of the respective location metadata associated with the first question-answer pair;
determining similarity scores for portions of the vector embeddings based on similarities of the vector embeddings to the respective first numeric vector representation and the respective one or more second numeric vector representations;
selecting a highest-ranked portion of the portions of the vector embeddings based on a ranking of the portions of the vector embeddings by the similarity scores; and
determining the first answer based on the highest-ranked portion.
22. The method of claim 21 , wherein the similarity scores are determined using a similarity function, the similarity function comprising cosine similarity or Manhattan distance.
23. The method of claim 21 , wherein the respective first numeric vector representation, the respective one or more second numeric vector representations, and the vector embeddings are generated using a neural network.
24. The method of claim 23 , wherein the respective first numeric vector representation, the respective one or more second numeric vector representations, and the vector embeddings are further transformed using a cryptographic hash function.