Learned evaluation model for grading quality of natural language generation outputs
Systems and methods for automatic evaluation of the quality of NLG outputs. In some aspects of the technology, a learned evaluation model may be pretrained first using NLG model pretraining tasks, and then with further pretraining tasks using automatically generated synthetic sentence pairs. In some cases, following pretraining, the evaluation model may be further fine-tuned using a set of human-graded sentence pairs, so that it learns to approximate the grades allocated by the human evaluators.
1. A method of training a neural network, comprising:
generating, by one or more processors of a processing system, a plurality of synthetic sentence pairs, each synthetic sentence pair of the plurality of synthetic sentence pairs comprising an original passage of text and a modified passage of text;
generating, by the one or more processors, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
a first training signal of a plurality of training signals based on whether the given synthetic sentence pair was generated using backtranslation; and
one or more second training signals of the plurality of training signals based on a prediction from a backtranslation prediction model regarding a likelihood that one of the original passage of text or the modified passage of text of the given synthetic sentence pair could have been generated by backtranslating the other one of the original passage of text or the modified passage of text of the given synthetic sentence pair;
pretraining, by the one or more processors, the neural network to predict, for each given synthetic sentence pair of the plurality of synthetic sentence pairs, the plurality of training signals for the given synthetic sentence pair; and
fine-tuning, by the one or more processors, the neural network to predict, for each given human-graded sentence pair of a plurality of human-graded sentence pairs, a grade allocated by a human grader to the given human-graded sentence pair.
2. The method of claim 1 , further comprising:
pretraining, by the one or more processors, the neural network to predict a mask token in each of a plurality of masked language modeling tasks; and
pretraining, by the one or more processors, the neural network to predict, for each given next-sentence prediction task of a plurality of next-sentence prediction tasks, whether a second passage of text of the given next-sentence prediction task directly follows a first passage of text of the given next-sentence prediction task.
3. The method of claim 2 , further comprising:
generating, by the one or more processors, the plurality of masked language modeling tasks; and
generating, by the one or more processors, the plurality of next-sentence prediction tasks.
4. The method of claim 1 , wherein generating the plurality of synthetic sentence pairs comprises, for each given synthetic sentence pair of a first subset of the synthetic sentence pairs:
translating, by the one or more processors, the original passage of text of the given synthetic sentence pair from a first language into a second language, to create a translated passage of text; and
translating, by the one or more processors, the translated passage of text from the second language into the first language, to create the modified passage of text of the given synthetic sentence pair.
5. The method of claim 4 , wherein generating the plurality of synthetic sentence pairs comprises, for each given synthetic sentence pair of a second subset of the synthetic sentence pairs, substituting one or more words of the original passage of text of the given synthetic sentence pair to create the modified passage of text of the given synthetic sentence pair.
6. The method of claim 5 , wherein generating the plurality of synthetic sentence pairs further comprises, for each given synthetic sentence pair of a third subset of the synthetic sentence pairs, removing one or more words of the original passage of text of the given synthetic sentence pair to create the modified passage of text of the given synthetic sentence pair.
7. The method of claim 1 , further comprising generating, by the one or more processors, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
one or more third training signals of the plurality of training signals based on one or more scores generated by comparing the original passage of text of the given synthetic sentence pair to the modified passage of text of the given synthetic sentence pair using one or more automatic metrics.
8. The method of claim 7 , wherein the one or more automatic metrics includes at least one of the BLEU metric, the ROUGE metric, or the BERTscore metric.
9. The method of claim 7 , further comprising generating, by the one or more processors, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
one or more fourth training signals of the plurality of training signals based on a prediction from a textual entailment model regarding a likelihood that the modified passage of text of the given synthetic sentence pair entails or contradicts the original passage of text of the given synthetic sentence pair.
10. The method of claim 1 , further comprising generating, by the one or more processors, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
one or more fourth training signals of the plurality of training signals based on a prediction from a textual entailment model regarding a likelihood that the modified passage of text of the given synthetic sentence pair entails or contradicts the original passage of text of the given synthetic sentence pair.
11. A processing system comprising:
a memory; and
one or more processors coupled to the memory and configured to:
generate a plurality of synthetic sentence pairs, each synthetic sentence pair of the plurality of synthetic sentence pairs comprising an original passage of text and a modified passage of text;
generate, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
a first training signal of a plurality of training signals based on whether the given synthetic sentence pair was generated using backtranslation; and
one or more second training signals of the plurality of training signals based on a prediction from a backtranslation prediction model regarding a likelihood that one of the original passage of text or the modified passage of text of the given synthetic sentence pair could have been generated by backtranslating the other one of the original passage of text or the modified passage of text of the given synthetic sentence pair;
pretrain a neural network to predict, for each given synthetic sentence pair of the plurality of synthetic sentence pairs, the plurality of training signals for the given synthetic sentence pair; and
fine-tune the neural network to predict, for each given human-graded sentence pair of a plurality of human-graded sentence pairs, a grade allocated by a human grader to the given human-graded sentence pair.
12. The system of claim 11 , wherein the one or more processors are further configured to:
pretrain the neural network to predict a mask token in each of a plurality of masked language modeling tasks; and
pretrain the neural network to predict, for each given next-sentence prediction task of a plurality of next-sentence prediction tasks, whether a second passage of text of the given next-sentence prediction task directly follows a first passage of text of the given next-sentence prediction task.
13. The system of claim 12 , wherein the one or more processors are further configured to:
generate the plurality of masked language modeling tasks; and
generate the plurality of next-sentence prediction tasks.
14. The system of claim 11 , wherein the one or more processors being configured to generate the plurality of synthetic sentence pairs comprises being configured to, for each given synthetic sentence pair of a first subset of the synthetic sentence pairs:
translate the original passage of text of the given synthetic sentence pair from a first language into a second language, to create a translated passage of text; and
translate the translated passage of text from the second language into the first, language, to create the modified passage of text of the given synthetic sentence pair.
15. The system of claim 14 , wherein the one or more processors being configured to generate the plurality of synthetic sentence pairs further comprises being configured to, for each given synthetic sentence pair of a second subset of the synthetic sentence pairs, substitute one or more words of the original passage of text of the given synthetic sentence pair to create the modified passage of text of the given synthetic sentence pair.
16. The system of claim 15 , wherein the one or more processors being configured to generate the plurality of synthetic sentence pairs further comprises being configured to, for each given synthetic sentence pair of a third subset of the synthetic sentence pairs, remove one or more words of the original passage of text of the given synthetic sentence pair to create the modified passage of text of the given synthetic sentence pair.
17. The system of claim 11 , wherein the one or more processors are further configured to generate, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
one or more third training signals of the plurality of training signals based on one or more scores generated by comparing the original passage of text of the given synthetic sentence pair to the modified passage of text of the given synthetic sentence pair using one or more automatic metrics.
18. The system of claim 17 , wherein the one or more automatic metrics includes at least one of the BLEU metric, the ROUGE metric, or the BERTscore metric.
19. The system of claim 17 , wherein the one or more processors are further configured to generate, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
one or more fourth training signals of the plurality of training signals based on a prediction from a textual entailment model regarding a likelihood that the modified passage of text of the given synthetic sentence pair entails or contradicts the original passage of text of the given synthetic sentence pair.
20. The system of claim 11 , wherein the one or more processors are further configured to generate, for each given synthetic sentence pair of the plurality of synthetic sentence pairs:
one or more fourth training signals of the plurality of training signals based on a prediction from a textual entailment model regarding a likelihood that the modified passage of text of the given synthetic sentence pair entails or contradicts the original passage of text of the given synthetic sentence pair.