Methods, mediums, and systems for reusable intelligent search workflows
Exemplary embodiments provide methods, mediums, and systems for performing a reusable, intelligent semantic search across a potentially large number of records. Embodiments may be particularly useful for responding to requests for information from regulatory agencies. In one embodiment, an embedding model is trained to embed queries in an embedding space. When a new query is received, the new query is embedded with the embedding model. A set of documents (e.g., previous responses to regulatory inquiries) may be searched using the embedded query and an indexing model that allows for efficient searches of embedding spaces. A number of results may be returned from the document store, and the results may be ranked by a ranking model. User feedback about the quality of the results may be received, and the ranking model may be retrained based on the feedback.
1. A computer-implemented method comprising, via at least one computing device comprising at least one processor and a memory storing instructions that, when executed by the at least one processor, cause the computing device to perform the computer-implemented method via:
receiving a natural language search query pertaining to a request for information;
accessing a set of documents, each document of the set of documents comprising a document query and a document response, each document query embedded in an embedding space according to an embedding model, wherein the set of documents are clustered into a plurality of cells based on the document query;
creating a search query embedding of the search query using the embedding model;
assigning the search query embedding to a cell of the plurality of cells;
using the search query embedding to perform a semantic search on the embedding space of documents clustered in the cell; and
returning at least one returned document of the set of documents as a search result based on a proximity of the document query from the at least one returned document to the search query in the embedding space, wherein the returned document along with metadata associated with the returned document includes an identifier for an individual that generated or approved the returned document is displayed in a graphical user interface (GUI).
2. The computer-implemented method of claim 1 , wherein the request for information is from a regulatory entity, and the set of documents comprises previous responses to at least one previous query from the regulatory entity; and
wherein the natural language search query is also displayed on the GUI.
3. The computer-implemented method of claim 1 , wherein the at least one returned document comprises a plurality of returned documents, each of the plurality of returned documents being ranked based on the proximity; and
wherein the GUI is configured to display the plurality of returned documents in a ranked order based on the proximity, the method further comprising:
submitting the document query from each of the plurality of returned documents to a reranking model configured to rerank the plurality of returned documents based on user feedback provided in the GUI; and
reranking the plurality of returned documents based on the reranking model.
4. The computer-implemented method of claim 3 , further comprising:
receiving the user feedback for a selected search result; and
retraining the reranking model based on the user feedback.
5. The computer-implemented method of claim 1 , further comprising:
training the embedding model with a set of labeled data; and
training an indexing model configured to perform the semantic search on the embedding space.
6. The computer-implemented method of claim 5 , wherein the set of documents is a first set of documents and the semantic search is a first semantic
swapping the first set of documents for a second set of documents different from the first set of documents; and
applying the indexing model to perform a second semantic search based on a new natural language search query.
7. The computer-implemented method of claim 1 , wherein the proximity of the document query from the returned document to the search query in the embedding space is represented as a cosine similarity.
8. A non-transitory computer-readable storage medium, the computer-readable storage medium storing instructions that,-when executed by a computer, cause the computer to:
receive a natural language search query pertaining to a request for information;
access a set of documents, each document of the set of documents comprising a document query and a document response, each document query embedded in an embedding space according to an embedding model, wherein the set of documents are clustered into a plurality of cells based on the document query;
create a search query embedding of the search query using the embedding model;
assign the search query embedding to a cell of the plurality of cells;
use the search query embedding to perform a semantic search on the embedding space of documents clustered in the cell; and
return at least one returned document of the set of documents as a search result based on a proximity of the document query from the at least one returned document to the search query in the embedding space, wherein the returned document along with metadata associated with the returned document includes an identifier for an individual that generated or approved the returned document is displayed in a graphical user interface (GUI).
9. The computer-readable storage medium of claim 8 , wherein the request for information is from a regulatory entity, and the set of documents comprises previous responses to at least one previous query from the regulatory entity; and
wherein the natural language search query is also displayed on the GUI.
10. The computer-readable storage medium of claim 8 , wherein the at least one returned document comprises a plurality of returned documents, each of the plurality of returned documents being ranked based on the proximity;
wherein the GUI is configured to display the plurality of returned documents in a ranked order based on the proximity; and
wherein the instructions further configure the computer to:
submit the document query from each of the plurality of returned documents to a reranking model configured to rerank the plurality of returned documents based on user feedback provided on the GUI; and
rerank the plurality of returned documents based on the reranking model.
11. The computer-readable storage medium of claim 10 , wherein the instructions further configure the computer to:
receive the user feedback for a selected search result; and
retrain the reranking model based on the user feedback.
12. The computer-readable storage medium of claim 8 , wherein the instructions further configure the computer to:
train the embedding model with a set of labeled data; and
train an indexing model configured to perform the semantic search on the embedding space.
13. The computer-readable storage medium of claim 12 , wherein the set of documents is a first set of documents and the semantic search is a first semantic search, and wherein the instructions further configure the computer to:
swap the first set of documents for a second set of documents different from the first set of documents; and
apply the indexing model to perform a second semantic search based on a new natural language search query.
14. The computer-readable storage medium of claim 8 , wherein the proximity of the document query from the returned document to the search query in the embedding space is represented as a cosine similarity.
15. A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, cause the apparatus to:
receive a natural language search query pertaining to a request for information;
access a set of documents, each document of the set of documents comprising a document query and a document response, each document query embedded in an embedding space according to an embedding model, wherein the set of documents are clustered into a plurality of cells based on the document query;
create a search query embedding of the search query using the embedding model;
assign the search query embedding to a cell of the plurality of cells;
use the search query embedding to perform a semantic search on the embedding space of documents clustered in the cell; and
return at least one returned document of the set of documents as a search result based on a proximity of the document query from the at least one returned document to the search query in the embedding space, wherein the returned document along with metadata associated the returned document includes an identifier for an individual that generated or approved the returned document is displayed in a graphical user interface (GUI).
16. The computing apparatus of claim 15 , wherein the request for information is from a regulatory entity, and the set of documents comprises previous responses to at least one previous query from the regulatory entity; and
wherein the natural language search query is also displayed on the GUI.
17. The computing apparatus of claim 15 , wherein the at least one returned document comprises a plurality of returned documents, each of the plurality of returned documents being ranked based on the proximity;
wherein the GUI is configured to display the plurality of returned documents in a ranked order based on the proximity; and
wherein the instructions further configure the apparatus to:
submit the document query from each of the plurality of returned documents to a reranking model configured to rerank the plurality of returned documents based on user feedback provided on the GUI; and
rerank the plurality of returned documents based on the reranking model.
18. The computing apparatus of claim 17 , wherein the instructions further configure the apparatus to:
receive the user feedback for a selected search result; and
retrain the reranking model based on the user feedback.
19. The computing apparatus of claim 15 , wherein the instructions further configure the apparatus to:
train the embedding model with a set of labeled data; and
train an indexing model configured to perform the semantic search on the embedding space.
20. The computing apparatus of claim 19 , wherein the set of documents is a first set of documents and the semantic search is a first semantic search, and wherein the instructions further configure the apparatus to:
swap the first set of documents for a second set of documents different from the first set of documents; and
apply the indexing model to perform a second semantic search based on a new natural language search query.