Weakly supervised semantic entity recognition using general and target domain knowledge
Methods and systems for performing semantic entity recognition. The method includes accessing a document stored in a memory and selecting from a general knowledge data repository, target domain information based on a specified target domain. The method also includes generating a plurality of weak annotators for the document based upon the selected target domain information and expert knowledge from a domain-specific expert knowledge data repository and applying the plurality of weak annotators to the document to generate a plurality of weak labels. The method further includes selecting at least one weak label from the plurality of weak labels as training data and training a semantic entity prediction model using the training data.
1. A system for performing semantic entity recognition, the system comprising
a general knowledge data repository,
a domain-specific expert knowledge data repository, and
an electronic processor configured to
access a document stored in a memory;
select, from the general knowledge data repository, target domain information based on a specified target domain;
generate a plurality of weak annotators for the document based upon the selected target domain information and expert knowledge from the domain-specific expert knowledge data repository;
apply the plurality of weak annotators to the document to generate a plurality of weak labels;
select at least one weak label from the plurality of weak labels as training data; and
train a semantic entity prediction model using the training data.
2. The system of claim 1 , wherein the electronic processor is further configured to pre-process the document to generate an unlabeled data set.
3. The system of claim 2 , wherein a plurality of potential semantic entities is generated using the unlabeled data set.
4. The system of claim 1 , wherein the specified target domain is associated with a domain of the document.
5. The system of claim 1 , wherein the electronic processor is further configured to combine at least two weak labels of the plurality of weak labels to generate the training data.
6. The system of claim 5 , wherein each of the plurality of weak labels is combined to generate the training data.
7. The system of claim 1 , wherein the semantic entity prediction model is a machine learning model.
8. A method for performing semantic entity recognition, the method comprising
accessing, with an electronic processor, a document stored in a memory;
selecting, with the electronic processor, from a general knowledge data repository, target domain information based on a specified target domain;
generating, with the electronic processor, a plurality of weak annotators for the document based upon the selected target domain information and expert knowledge from a domain-specific expert knowledge data repository;
applying, with the electronic processor, the plurality of weak annotators to the document to generate a plurality of weak labels;
selecting, with the electronic processor, at least one weak label from the plurality of weak labels as training data; and
training, with the electronic processor, a semantic entity prediction model using the training data.
9. The method of claim 8 , further comprising pre-processing, with the electronic processor, the document to generate an unlabeled data set.
10. The method of claim 9 , wherein a plurality of potential semantic entities is generated using the unlabeled data set.
11. The method of claim 8 , wherein the specified target domain is associated with a domain of the document.
12. The method of claim 8 , further including combining, with the electronic processor, at least two weak labels of the plurality of weak labels to generate the training data.
13. The method of claim 12 , wherein each of the plurality of weak labels is combined to generate the training data.
14. The method of claim 8 , wherein the semantic entity prediction model is a machine learning model.
15. A non-transitory, computer-readable medium containing instructions that, when executed by an electronic processor, are configured to perform a set of functions, the set of functions including
accessing a document stored in a memory;
selecting from a general knowledge data repository, target domain information based on a specified target domain;
generating a plurality of weak annotators for the document based upon the selected target domain information and expert knowledge from a domain-specific expert knowledge data repository;
applying the plurality of weak annotators to the document to generate a plurality of weak labels;
selecting at least one weak label from the plurality of weak labels as training data; and
training a semantic entity prediction model using the training data.
16. The non-transitory, computer readable medium of claim 15 , wherein the set of functions further includes pre-processing the document to generate an unlabeled data set.
17. The non-transitory, computer-readable medium of claim 16 , wherein a plurality of potential semantic entities is generated using the unlabeled data set.
18. The non-transitory, computer readable medium of claim 15 , wherein the set of functions further includes combining at least two weak labels of the plurality of weak labels to generate the training data.
19. The non-transitory, computer readable medium of claim 18 , wherein each of the plurality of weak labels is combined to generate the training data.
20. The non-transitory, computer readable medium of claim 15 , wherein the semantic entity prediction model is a machine learning model.