Relation extraction using co-training with distant supervision
Generating, updating, and using a knowledge graph. Concepts in a knowledge graph can have relations to one another. These relations may be expressed as confidence values. A training data set may be split into two portions, with the first portion used to update confidence values for existing relations between concept pairs, using the knowledge graph. These confidence values can be used, together with the second portion used to update confidence values for known phrases that express known relations. These confidence values, in turn, can be used, together with the first portion, to increase the accuracy of the original confidence scores with respect to existing relations. The process may be iteratively employed, with each iteration increasing the accuracy of confidence scores.
1. A computer implemented method for updating an electronically stored knowledge graph of a knowledge domain, comprising:
receiving a natural language text comprising sentences;
splitting the sentences into at least a first portion and a second portion;
identifying concepts in the sentences of the first portion;
determining, for a first sentence corresponding to at least one sentence in the first portion having a concept pair, a likelihood that the first sentence exhibits an existing relation between concepts of the concept pair, wherein the determined likelihood represents a first confidence value; and
determining, for a second sentence corresponding to at least one sentence in the second portion having the concept pair, a likelihood that a word or phrase connecting concepts of the concept pair exhibits the existing relation, wherein the determined likelihood represents a second confidence value and is based, in part, on the first confidence value;
determining, for a third sentence corresponding to at least one sentence in the first portion having the concept pair, a likelihood that the third sentence exhibits the existing relation, wherein the determined likelihood represents a third confidence value, and is based, in part, on the second confidence value;
iteratively determining successive likelihoods according to the first, second, and third confidence values, by alternating between sentences of the first portion and sentences of the second portion until the successive likelihoods reach corresponding threshold confidence values;
updating the reference knowledge graph to include relations between concepts whose corresponding confidence scores exceed a threshold value; and
using the updated reference knowledge graph in an analysis of additional natural language text.
2. The method of claim 1 , further comprising labeling the sentences, the labeling comprising:
identifying concepts in the sentences text;
annotating the sentences with the concepts;
and extracting from the natural language text, lexical and syntactic features.
3. The method of claim 1 , wherein determining, for at least one sentence in the first portion having a concept pair, a likelihood that the sentence is an instance of the concepts in the concept pair exhibiting an existing relation, comprises:
identifying as the existing relation, an existing relation between respective categories of the concepts in the concept pair as defined by a reference knowledge graph.
4. The method of claim 1 , wherein determining, for at least one sentence in the first portion having a concept pair, a likelihood that the sentence is an instance of the concepts in the concept pair exhibiting an existing relation, is based on identifying one or more concept pairs in the at least one sentence, the identifying comprising:
comparing words of the at least one sentence in the first portion to elements of a knowledge graph;
identifying matching words as concepts; and
pairing at least two of the identifying concepts with one another to form a concept pair.
5. The method of claim 1 , wherein determining, for at least one sentence in the first portion having a concept pair, a likelihood that the sentence is an instance of the concepts in the concept pair exhibiting an existing relation, comprises:
identifying as the existing relation, an existing relation between the concepts in the concept pair as defined by a reference knowledge graph.
6. The method of claim 1 , further comprising:
updating a reference knowledge graph to include relations between concepts whose corresponding confidence scores exceed a threshold value.
7. A computer program product, comprising a non-transitory tangible storage device having program code embodied therewith, the program code executable by a processor of a computer to perform a method, the method comprising:
receiving, by the processor, a natural language text comprising sentences;
splitting, by the processor, the sentences into at least a first portion and a second portion;
identifying, by the processor, concepts in the sentences of the first portion;
determining, by the processor, for a first sentence corresponding to at least one sentence in the first portion having a concept pair, a likelihood that the first sentence exhibits an existing relation between concepts of the concept pair, wherein the determined likelihood represents a first confidence value; and
determining, by the processor, for a second sentence corresponding to at least one sentence in the second portion having the concept pair, a likelihood that a word or phrase connecting concepts of the concept pair exhibits the existing relation, wherein the determined likelihood represents a second confidence value and is based, in part, on the first confidence value;
determining, for a third sentence corresponding to at least one sentence in the first portion having the concept pair, a likelihood that the third sentence exhibits the existing relation, wherein the determined likelihood represents a third confidence value, and is based, in part, on the second confidence value;
iteratively determining successive likelihoods according to the first, second, and third confidence values, by alternating between sentences of the first portion and sentences of the second portion until the successive likelihoods reach corresponding threshold confidence values;
updating the reference knowledge graph to include relations between concepts whose corresponding confidence scores exceed a threshold value; and
using the updated reference knowledge graph in an analysis of additional natural language text.
8. The computer program product of claim 7 , further comprising labeling the sentences, the labeling comprising:
identifying concepts in the sentences text;
annotating the sentences with the concepts;
and extracting from the natural language text, lexical and syntactic features.
9. The computer program product of claim 7 , wherein determining, for at least one sentence in the first portion having a concept pair, a likelihood that the sentence is an instance of the concepts in the concept pair exhibiting an existing relation, comprises:
identifying, by the processor, as the existing relation, an existing relation between respective categories of the concepts in the concept pair as defined by a reference knowledge graph.
10. The computer program product of claim 7 , wherein determining, for at least one sentence in the first portion having a concept pair, a likelihood that the sentence is an instance of the concepts in the concept pair exhibiting an existing relation, is based on identifying one or more concept pairs in the at least one sentence, the identifying comprising:
comparing, by the processor, words of the at least one sentence in the first portion to elements of a knowledge graph;
identifying, by the processor, matching words as concepts; and
pairing, by the processor, at least two of the identifying concepts with one another to form a concept pair.
11. The computer program product of claim 7 , wherein determining, for at least one sentence in the first portion having a concept pair, a likelihood that the sentence is an instance of the concepts in the concept pair exhibiting an existing relation, comprises:
identifying, by the processor, as the existing relation, an existing relation between the concepts in the concept pair as defined by a reference knowledge graph.
12. The computer program of claim 7 , further comprising:
updating, by the processor, a reference knowledge graph to include relations between concepts whose corresponding confidence scores exceed a threshold value.
13. A computer system, comprising:
one or more computer devices each having one or more processors and one or more tangible storage devices; and
a program embodied on at least one of the one or more storage devices, the program having a plurality of program instructions for execution by the one or more processors, the program instructions comprising instructions for:
receiving a natural language text comprising sentences;
splitting the sentences into at least a first portion and a second portion;
identifying concepts in the sentences of the first portion;
determining, for a first sentence corresponding to at least one sentence in the first portion having a concept pair, a likelihood that the first sentence exhibits an existing relation between concepts of the concept pair, wherein the determined likelihood represents a first confidence value; and
determining, for a second sentence corresponding to at least one sentence in the second portion having the concept pair, a likelihood that a word or phrase connecting concepts of the concept pair exhibits the existing relation, wherein the determined likelihood represents a second confidence value and is based, in part, on the first confidence value;
determining, for a third sentence corresponding to at least one sentence in the first portion having the concept pair, a likelihood that the third sentence exhibits the existing relation, wherein the determined likelihood represents a third confidence value, and is based, in part, on the second confidence value;
iteratively determining successive likelihoods according to the first, second, and third confidence values, by alternating between sentences of the first portion and sentences of the second portion until the successive likelihoods reach corresponding threshold confidence values;
updating the reference knowledge graph to include relations between concepts whose corresponding confidence scores exceed a threshold value; and
using the updated reference knowledge graph in an analysis of additional natural language text.
14. The system of claim 13 , the program instructions further comprising instructions for labeling the sentences, comprising instructions for:
identifying concepts in the sentences text;
annotating the sentences with the concepts;
and extracting from the natural language text, lexical and syntactic features.