Data processing device, data processing method, and data processing program
A data processing device includes processing circuitry configured to extract a second word corresponding to a first word included in first text from among a plurality of words belonging to a predetermined domain, repeat processing in the extraction for all words included in the first text to generate a confusion network that expresses a plurality of sentence possibilities with one network configuration and is an expression format of a word sequence, and search for a grammatically correct word string in the confusion network using a language model that evaluates grammatical correctness of the word string, and select a word string to be output.
1. A data processing device comprising:
processing circuitry configured to:
use a replacement model, that estimates a domain-dependent word probability distribution at a position where a word appears in a sentence, to estimate a word probability distribution obtained by replacing every first word of a plurality of first words included in text of a first domain with a second word of a second domain;
sample values corresponding to a vocabulary size from a Gumbel distribution;
add the values to the word probability distribution to generate a new word probability distribution;
repeat the generation of the new word probability distribution a plurality of times for all of the plurality of first words included in the text of the first domain, select a plurality of the second words from a top of the new word probability distribution for every first word of the plurality of first words, and arrange the selected second words to be ordered by position to generate a confusion network that expresses a plurality of sentence possibilities with one network configuration and is an expression format of a word sequence;
search for a grammatically correct word string in the confusion network using a language model that evaluates grammatical correctness of the word string; and
select a word string to be output,
wherein the language model is trained based on learning data,
wherein the data processing device adds the selected word string to the learning data to enrich the learning data,
wherein the language model is further trained using the enriched learning data, and
wherein the processing circuitry is configured to use the further trained language model to search for the grammatically correct word string in the confusion network.
2. The data processing device according to claim 1 ,
wherein the processing circuitry is further configured to preferentially output either the grammatically correct word string or a word string having diversity according to a preset first parameter indicating a degree of priority between the grammatically correct word string and the word string having diversity.
3. The data processing device according to claim 2 , wherein a value of the first parameter is adjusted according to a second parameter that sets a number of words indicated in each segment of the confusion network.
4. A data processing method executed by a data processing device, the data processing method comprising:
using a replacement model, that estimates a domain-dependent word probability distribution at a position where a word appears in a sentence, to estimate a word probability distribution obtained by replacing every first word of a plurality of first words included in text of a first domain with a second word of a second domain, sampling values corresponding to a vocabulary size from a Gumbel distribution, and adding the values to the word probability distribution to generate a new word probability distribution;
repeating the generation of the new word probability distribution a plurality of times for all of the plurality of first words included in the text of the first domain, select a plurality of the second words from a top of the new word probability distribution for every first word of the plurality of first words, and arrange the selected second words to be ordered by position to generate a confusion network that expresses a plurality of sentence possibilities with one network configuration and is an expression format of a word sequence; and
searching for a grammatically correct word string in the confusion network using a language model that evaluates grammatical correctness of the word string; and
selecting a word string to be output,
wherein the language model is trained based on learning data,
wherein the method further comprises adding the selected word string to the learning data to enrich the learning data,
wherein the language model is further trained using the enriched learning data, and
wherein the method further comprises using the further trained language model to search for the grammatically correct word string in the confusion network.
5. A non-transitory computer-readable recording medium storing therein a data processing program that causes a computer to execute a process comprising:
using a replacement model, that estimates a domain-dependent word probability distribution at a position where a word appears in a sentence, to estimate a word probability distribution obtained by replacing every first word of a plurality of first words included in text of a first domain with a second word of a second domain, sampling values corresponding to a vocabulary size from a Gumbel distribution, and adding the values to the word probability distribution to generate a new word probability distribution;
repeating the generation of the new word probability distribution a plurality of times for all of the plurality of first words included in the text of the first domain, select a plurality of the second words from a top of the new word probability distribution for every first word of the plurality of first words, and arrange the selected second words to be ordered by position to generate a confusion network that expresses a plurality of sentence possibilities with one network configuration and is an expression format of a word sequence; and
searching for a grammatically correct word string in the confusion network using a language model that evaluates grammatical correctness of the word string, and
selecting a word string to be output,
wherein the language model is trained based on learning data,
wherein the process further comprises adding the selected word string to the learning data to enrich the learning data,
wherein the language model is further trained using the enriched learning data, and
wherein the process further comprises using the further trained language model to search for the grammatically correct word string in the confusion network.
6. The data processing device according to claim 1 , wherein the first domain is different from the second domain.
7. The data processing device according to claim 6 , wherein
the first domain includes text in a formal style, and
the second domain includes text in a conversational style.
8. The data processing device according to claim 1 , wherein
the first domain includes text in a formal style, and
the second domain includes text in a colloquial style.
9. The data processing device according to claim 1 , wherein domain corresponds to a category defined by a content of a sentence, a style of the sentence, or a combination thereof.
10. The data processing device according to claim 1 , wherein the replacement model is a bidirectional long short-term memory (LSTM) model configured to estimate the domain-dependent word probability distribution by processing a forward partial word string of the text of the first domain and a backward partial word string of the text of the first domain.
11. The data processing device according to claim 10 , wherein the replacement model generates a domain-dependent hidden state vector by concatenating hidden state vectors from a forward LSTM layer, a backward LSTM layer, and a binary domain label indicating the second domain.
12. The data processing device according to claim 1 , wherein the processing circuitry is configured to select the plurality of the second words from the top of the new word probability distribution such that a cumulative probability of the selected second words does not exceed a preset threshold.
13. The data processing device according to claim 12 , wherein the processing circuitry is configured to normalize probabilities of the selected second words such that a total probability equals one for each position in the confusion network.
14. The data processing device according to claim 1 , wherein the replacement model is trained using learning data from both the first domain and the second domain.
15. The data processing device according to claim 1 , wherein the processing circuitry is configured to select the plurality of the second words from a top K words of the new word probability distribution, where K is a preset parameter determining a number of words per segment of the confusion network.