Learning apparatus, learning method and program
A learning apparatus includes a memory and at least one processor connected to the memory, wherein the processor configured to: convert input text data into a feature amount sequence based on a language model; and update parameters of the language model based on the text data, the feature amount sequence, and a word vector learned in advance.
1 . A learning apparatus comprising:
a memory; and
at least one processor connected to the memory,
wherein the processor configured to:
convert input text data into a feature amount sequence of the input text data based on a language model, wherein the language model is pre-trained in a first subject domain; and
update, as training of the language model, parameters of the language model based on the text data, the feature amount sequence of the input text data, and a word vector,
wherein the word vector is learned in advance in a second subject domain, the second subject domain is a part of the first subject domain, the word vector represents semantics of a word in the second subject domain, thereby the language model with the updated parameters as trained performing a target task of the second subject domain with accuracy, and
in a case where the word vector includes word vectors of subwords obtained by dividing a word, and the language model divides the text data to have a finer granularity than a word, update the parameters of the language model based on the word vectors of the subwords.
2 . A learning apparatus comprising:
a memory; and
at least one processor connected to the memory,
wherein the processor configured to:
convert input text data into a feature amount sequence of the input text data based on a language model, wherein the language model is pre-trained in a first subject domain; and
update, as training of the language model, parameters of the language model based on the text data, the feature amount sequence of the input text data, and a word vector,
wherein the word vector is learned in advance in a second subject domain, the second subject domain is a part of the first subject domain, the word vector represents semantics of a word in the second subject domain, thereby the language model with the updated parameters as trained performing a target task of the second subject domain with accuracy, and
in a case where the language model divides the text data to have a finer granularity than a word, update the parameters of the language model based on an average value of intermediate representations of the language model and the word vector for subwords belonging to a same word.
3 . A learning method to be executed by a computer including a memory and at least one processor connected to the memory, the method comprising:
converting input text data into a feature amount sequence of the input text data based on a language model, wherein the language model is pre-trained in a first subject domain; and
updating, as training of the language model, parameters of the language model based on the text data, the feature amount sequence of the input text data, and a word vector,
wherein the word vector is learned in advance in a second subject domain, the second subject domain is a part of the first subject domain, the word vector represents semantics of a word in the second subject domain, thereby the language model with the updated parameters as trained performing a target task of the second subject domain with accuracy, and
in a case where the word vector includes word vectors of subwords obtained by dividing a word, and the language model divides the text data to have a finer granularity than a word, update the parameters of the language model based on the word vectors of the subwords.
4 . A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which, when executed, cause a computer to function as the learning apparatus according to claim 1 .
5 . The learning apparatus according to claim 1 ,
wherein the word is at least a part of the input text data.
6 . The learning apparatus according to claim 1 ,
wherein the word vector is learned in advance by using another corpus,
said another corpus comprises more text data than the corpus, thereby the word vector capturing a knowledge that the pre-trained language model lacks.
7 . The learning apparatus according to claim 1 ,
wherein the word vector is learned in advance based on a general corpus, and
an amount of the corpus is greater than a total amount of the input text data.
8 . The learning apparatus according to claim 1 ,
wherein a domain of the input text data is the same as a domain of text data to be processed in a target task.