Phrase generation relationship estimation model learning device, phrase generation device, method, and program
The present disclosure relates to concurrent learning of a relationship estimation model and a phrase generation model. The relationship estimation model estimates a relationship between phrases. The phrase generation model generates a phrase that relates to an input phrase. The phrase generation model includes an encoder and a decoder. The encoder converts a phrase into a vector using a three-piece set as learning data. The decoder generates, based on the converted vector and a connection expression or a relationship label, a phrase having a relationship expressed by the connection expression or the relationship label for the phrase. The relationship estimation model generates a relationship score from the converted vector, which indicates each phrase included in a combination of the phrases, and a vector indicating the connection expression and the relationship label.
1. A computer-implemented method for processing phrases, the method comprising:
receiving an input text, wherein the input text includes a plurality of phrases;
extracting, based on a dependency analysis, a triplet, wherein the triplet comprises a first phrase, a second phrase, and a dependency label, and wherein the dependency label defines a first dependency relationship between the first phrase and the second phrase;
generating, by an encoder, a set of vectors, wherein the set of vectors is for training a relationship estimation model, and wherein the set of vectors comprises:
a first vector based on the first phrase,
a second vector based on the second phrase, and
a third vector based on the dependency label;
generating, by the encoder, a first pair, wherein the first pair is for training a phrase generation model, wherein the relationship estimation model and the phase generation model share at least a part of a common neural network, and wherein the first pair comprises the first vector and the third vector;
generating, by the encoder a second pair, wherein the second pair is for training the phrase generation model, wherein the second pair comprises the second vector and a fourth vector, and wherein the fourth vector is based on a reverse label of the first dependency relationship; and
generating, based on the set of vectors for training, the relationship estimation model;
using the first pair and the second pair as correct learning data, training the phrase generation model;
receiving a third phrase and a connection expression;
determining, based on the third phrase and the connection expression using the trained phrase generation model, a fourth phrase, wherein the phrase generation model once trained generates a phrase from a pair of an input phrase and an input dependency label, and wherein the fourth phrase relates to the third phrase based on the connection expression;
determining, based on the trained relationship estimation model, a relation score, wherein the trained relationship estimation model once trained estimates a type of dependency relationship between a pair of input phrases, and wherein the relation score defines a degree of a second dependency relationship between the received third phrase and the determined fourth phrase; and
providing the fourth phrase and the determined relation score in response to the received third phrase and the connection expression.
2. The computer-implemented method of claim 1 , wherein the phrase generation model is based on an attention-based encode-decoder model, and the method further comprising:
training, based on the second pair for training, the phrase generation model; and
updating, based at least on a loss function relating to encoder-decoder, one or more parameters of the phrase generation model.
3. The computer-implemented method of claim 1 , wherein the phrase generation model includes a decoder, wherein the decoder generates a phrase having a relationship represented by the dependency label of the triplet.
4. The computer-implemented method of claim 1 , the method further comprising:
concurrently performing, for training:
generating, based on the set of vectors, the relationship estimation model; and
generating, based the first pair and the second pair, the phrase generation model.
5. The computer-implemented method of claim 1 , wherein the generating the relationship estimation model and the generating the phrase generation model relates to minimizing a loss function, wherein the loss function is based on a combination of a first loss function for the relationship estimation model and a second loss function for the phrase generation model.
6. The computer-implemented method of claim 1 , the method further comprising:
generating a negative example of the triplet for training; and
generating, based on the negative example of the triplet for training, the relationship estimation model.
7. A system for processing phrases, the system comprises:
a processor; and
a memory storing computer-executable instructions that when executed by the processor cause the system to:
receive an input text, wherein the input text includes a plurality of phrases;
extract, based on a dependency analysis, a triplet, wherein the triplet comprises a first phrase, a second phrase, and a dependency label, and wherein the dependency label defines a first dependency relationship between the first phrase and the second phrase;
generate, by an encoder, a set of vectors, wherein the set of vectors is for training a relationship estimation model, wherein the relationship estimation model and the phase generation model share at least a part of a common neural network, and wherein the set of vectors comprises:
a first vector based on the first phrase,
a second vector based on the second phrase, and
a third vector based on the dependency label;
generate, by the encoder, a first pair, wherein the first pair is for training a phrase generation model, wherein the relationship estimation model and the phase generation model share at least a part of a common neural network, and wherein the first pair comprises the first vector and the third vector;
generate, by the encoder a second pair, wherein the second pair is for training the phrase generation model, wherein the second pair comprises the second vector and a fourth vector, and wherein the fourth vector is based on a reverse label of the first dependency relationship; and
generate, based on the set of vectors for training, the relationship estimation model;
using the first pair and the second pair as correct learning data, train the phrase generation model;
receive a third phrase and a connection expression;
determine, based on the third phrase and the connection expression using the trained phrase generation model, a fourth phrase, wherein the phrase generation model once trained generates a phrase from a pair of an input phrase and an input dependency label, and wherein the fourth phrase relates to the third phrase based on the connection expression;
determine, based on the trained relationship estimation model, a relation score, wherein the trained relationship estimation model once trained estimates a type of dependency relationship between a pair of input phrases, and wherein the relation score defines a degree of a second dependency relationship between the received third phrase and the determined fourth phrase; and
provide the fourth phrase and the determined relation score in response to the received third phrase and the connection expression.
8. The system of claim 7 , wherein the phrase generation model is based on an attention-based encode-decoder model, and the computer-executable instructions when executed further causing the system to:
train, based on the second pair for training, the phrase generation model; and
update, based at least on a loss function relating to encoder-decoder, one or more parameters of the phrase generation model.
9. The system of claim 7 , wherein the phrase generation model includes a decoder, wherein the decoder generates a phrase having a relationship represented by the dependency label of the triple.
10. The system of claim 7 , the computer-executable instructions when executed further causing the system to:
concurrently perform, for training:
generating, based on the triple, the relationship estimation model; and
generating, based the first pair and the second pair, the phrase generation model.
11. The system of claim 7 , wherein the generating the relationship estimation model and the generating the phrase generation model relates to minimizing a loss function, wherein the loss function is based on a combination of a first loss function for the relationship estimation model and a second loss function for the phrase generation model.
12. The system of claim 7 , the computer-executable instructions when executed further causing the system to:
generate a negative example of the triple for training; and
generate, based on the negative example of the triple for training, the relationship estimation model.
13. A computer-readable non-transitory recording medium storing computer-executable instructions that when executed by a processor cause a computer system to:
receive an input text, wherein the input text includes a plurality of phrases;
extract, based on a dependency analysis, a triplet, wherein the triplet comprises a first phrase, a second phrase, and a dependency label, and wherein the dependency label defines a first dependency relationship between the first phrase and the second phrase;
generate, by an encoder, a set of vectors, wherein the set of vectors is for training a relationship estimation model, and wherein the set of vectors comprises:
a first vector based on the first phrase,
a second vector based on the second phrase, and
a third vector based on the dependency label;
generate, by the encoder, a first pair, wherein the first pair is for training a phrase generation model, wherein the relationship estimation model and the phase generation model share a at least a part of a common neural network, and wherein the first pair comprises the first vector and the third vector;
generate, by the encoder a second pair, wherein the second pair is for training the phrase generation model, wherein the second pair comprises the second vector and a fourth vector, and
wherein the fourth vector is based on a reverse label of the first dependency relationship; and
generate, based on the set of vectors for training, the relationship estimation model;
using the first pair and the second pair as correct learning data, train the phrase generation model;
receive a third phrase and a connection expression;
determine, based on the third phrase and the connection expression using the trained phrase generation model, a fourth phrase, wherein the phrase generation model once trained generates a phrase from a pair of an input phrase and an input dependency label, and wherein the fourth phrase relates to the third phrase based on the connection expression;
determine, based on the trained relationship estimation model, a relation score, wherein the trained relationship estimation model once trained estimates a type of dependency relationship between a pair of input phrases, and wherein the relation score defines a degree of a second dependency relationship between the received third phrase and the determined fourth phrase; and
provide the fourth phrase and the determined relation score in response to the received third phrase and the connection expression.
14. The computer-readable non-transitory recording medium of claim 13 , wherein the phrase generation model is based on an attention-based encode-decoder model, and the computer-executable instructions when executed further causing the system to:
train, based on the second pair for training, the phrase generation model; and
update, based at least on a loss function relating to encoder-decoder, one or more parameters of the phrase generation model.
15. The computer-readable non-transitory recording medium of claim 13 , wherein the phrase generation model includes a decoder, wherein the decoder generates a phrase having a relationship represented by the dependency label of the triple.
16. The computer-readable non-transitory recording medium of claim 13 , the computer-executable instructions when executed further causing the system to:
concurrently perform, for training:
generating, based on the triple, the relationship estimation model; and
generating, based the first pair and the second pair, the phrase generation model.
17. The computer-readable non-transitory recording medium of claim 13 , the computer-executable instructions when executed further causing the system to:
generate a negative example of the triple for training; and
generate, based on the negative example of the triple for training, the relationship estimation model.