Virtual knowledge graph construction for zero-shot domain-specific document retrieval
Systems and methods for text processing are described. Embodiments of the present disclosure receive a query comprising a natural language expression; extract a plurality of mentions from the query; generate a relation vector between a pair of the plurality of mentions using a relation encoder network, wherein the relation encoder network is trained using a contrastive learning process where mention pairs from a same document are labeled as positive samples and mention pairs from different documents are labeled as negative samples; combine the plurality of mentions with the relation vector to obtain a virtual knowledge graph of the query; identify a document corresponding to the query by comparing the virtual knowledge graph of the query to a virtual knowledge graph of the document; and transmit a response to the query, wherein the response includes a reference to the document.
1. A method for text processing, comprising:
receiving a query comprising a natural language expression;
extracting a plurality of query mentions from the query;
generating a virtual knowledge graph of the query by masking each of the plurality of query mentions and generating, using a relation encoder network, a query relation vector between a pair of the plurality of query mentions;
identifying a virtual knowledge graph of a document including a document relation vector between a pair of a plurality of document mentions;
computing a graph similarity score between the virtual knowledge graph of the query and the virtual knowledge graph of the document based on the query relation vector, the document relation vector, and a match between the pair of the plurality of document mentions of the document relation vector and the pair of the plurality of query mentions of the query relation vector by computing a relation similarity between the query relation vector and the document relation vector based on the match; and
transmitting a response to the query based on the document and the graph similarity score.
2. The method of claim 1 , further comprising:
providing a document search interface;
receiving an input from a user via the document search interface, wherein the query is based on the input; and
displaying the response via the document search interface.
3. The method of claim 1 , further comprising:
identifying a text delimiter;
dividing the query into a plurality of spans based on the text delimiter;
computing an entity score for each of the plurality of spans; and
selecting the plurality of query mentions based on the entity score for each of the plurality of spans.
4. The method of claim 1 , further comprising:
determining that the pair of the plurality of query mentions corresponds to a same entity based on the query relation vector.
5. The method of claim 1 , further comprising:
identifying the pair from the plurality of query mentions, wherein the pair includes a first mention and a second mention;
masking the first mention and the second mention using a first entity token and a second entity token, respectively; and
inserting a head token adjacent to the first mention and a tail token adjacent to the second mention, wherein the query relation vector is generated based on the first entity token, the second entity token, the head token, and the tail token.
6. The method of claim 1 , wherein:
each of the plurality of query mentions comprises a document index, a start index, an end index, and a corresponding entity.
7. The method of claim 1 , wherein:
the virtual knowledge graph of the query includes a plurality of nodes corresponding to the plurality of query mentions and an edge corresponding to the query relation vector.
8. The method of claim 7 , wherein:
the edge comprises a head mention, a tail mention, and the query relation vector.
9. The method of claim 1 , further comprising:
identifying matching mentions in the query and the document based on the virtual knowledge graph of the query and the virtual knowledge graph of the document; and
computing the graph similarity score for the query and the document based on the matching mentions, wherein the document is identified based on the graph similarity score.
10. The method of claim 1 , further comprising:
computing a candidate knowledge graph for each of a plurality of candidate documents; and
comparing the virtual knowledge graph of the query to the candidate knowledge graph for each of the plurality of candidate documents, wherein the document is identified based on the comparison.
11. The method of claim 10 , further comprising:
performing a bag-of-words analysis on each document in a collection of documents; and
selecting the plurality of candidate documents from the collection of documents based on the bag-of-words analysis.
12. A method for training a machine learning model, comprising:
receiving a training set including a plurality of documents;
extracting a plurality of mentions from the plurality of documents;
generating, using a relation encoder network, a relation vector representing a relation between a pair of the plurality of mentions in each document; and
updating parameters of the relation encoder network based on the relation vector using a contrastive learning process where mention pairs from a same document are labeled as positive samples and mention pairs from different documents are labeled as negative samples.
13. The method of claim 12 , further comprising:
identifying a positive sample pair comprising a first mention pair and a second mention pair from a first document;
identifying a negative sample pair comprising the first mention pair from the first document and a third mention pair from a second document; and
computing a contrastive learning loss based on the positive sample pair and the negative sample pair, wherein the parameters of the relation encoder network are updated based on the contrastive learning loss.
14. The method of claim 13 , further comprising:
computing a first dot product of relation vectors representing the positive sample pair; and
computing a second dot product of relation vectors representing the negative sample pair, wherein the contrastive learning loss is computed based on the first dot product and the second dot product.
15. The method of claim 12 , further comprising:
receiving ground truth training data associating the pair of the plurality of mentions with a common entity; and
computing a cross-entropy loss based on the relation vector and the ground truth training data, wherein the parameters of the relation encoder network are updated based on the cross-entropy loss.
16. An apparatus for text processing, comprising:
at least one processor; and
at least one memory including instructions executable by the at least one processor to:
extract a plurality of query mentions from a query;
generate a virtual knowledge graph of the query by masking each of the plurality of query mentions and generating, using a relation encoder network, a query relation vector between a pair of the plurality of query mentions;
identify a virtual knowledge graph of a document including a document relation vector between a pair of a plurality of document mentions;
compute a graph similarity score between the virtual knowledge graph of the query and the virtual knowledge graph of the document based on the query relation vector, the document relation vector, and a match between the pair of the plurality of document mentions of the document relation vector and the pair of the plurality of query mentions of the query relation vector by computing a relation similarity between the query relation vector and the document relation vector based on the match; and
transmit a response to the query based on the document and the graph similarity score.
17. The apparatus of claim 16 , further comprising instructions executable by the at least one processor to:
compute a contrastive learning loss and to update parameters of the relation encoder network based on the contrastive learning loss.
18. The apparatus of claim 16 , further comprising instructions executable by the at least one processor to:
select a plurality of candidate documents from a collection of documents based on a bag-of-words analysis.
19. The apparatus of claim 16 , further comprising instructions executable by the at least one processor to:
receive the query from a user and to display the document.
20. The apparatus of claim 16 , wherein:
the relation encoder network comprises a transformer architecture.