Natural language generation
Devices and techniques are generally described for data-to-text generation. In various examples, a first machine learned model may receive first data including a structured representation of linguistic data. In various examples, the first machine learned model may generate first output data comprising a first natural language representation of the first data. In at least some examples, a second machine learning model may determine second data indicating that the first natural language representation is a semantically accurate representation of the first data. In some examples, the first output data may be selected for output based at least in part on the second data.
1. A method comprising:
receiving first audio data representing a question;
generating, by an automatic speech recognition component, first text data representing the first audio data;
receiving a first structured data representation of the first text data, the first structured data representation comprising triple data representing a response to the question, the triple data comprising first data representing a subject of the response, second data representing an object of the response, and third data representing a relationship between the subject and the object;
sending the triple data to a first machine learned model comprising a neural language model;
generating, by the first machine learned model, first output text comprising a first sentence representing the triple data;
training the first machine learned model using a training set comprising second text data and a structured data representation of the second text data;
determining, by a second machine learned model, first label data for the first output text, the first label data classifying the first sentence as a semantically accurate representation of the first structured data representation; and
generating, by a text-to-speech component, second audio data representing the first output text.
2. The method of claim 1 , further comprising:
generating, by the first machine learned model, second output text comprising a second sentence representing the triple data;
determining, by the second machine learned model, second label data for the second output text, the second label data classifying the second sentence as semantically inaccurate; and
selecting the first output text from among the first output text and the second output text for output as the second audio data by the text-to-speech component.
3. The method of claim 1 , further comprising:
generating, by the first machine learned model, second output text comprising a second sentence representing the triple data;
determining, by the second machine learned model, second label data for the second output text, the second label data classifying the second sentence as semantically accurate;
determining, by the first machine learned model, a first confidence score associated with the first output text;
determining, by the first machine learned model, a second confidence score associated with the second output text; and
selecting, using the first confidence score and the second confidence score, the first output text.
4. The method of claim 1 , further comprising:
receiving metadata including first metadata identifying the first data as the subject, second metadata identifying the second data as the object, and third metadata identifying the third data as the relationship between the subject and the object; and
using the metadata to generate the first output text such that the first data is included in the first output text.
5. A method comprising:
receiving first data comprising a structured representation of linguistic data;
determining first metadata describing a subject of the first data;
determining second metadata describing an object of the first data;
generating, by a first model comprising a computer-implemented neural language model, first output data comprising a first natural language representation of the first data, wherein the first natural language representation comprises a sentence comprising the subject and the object, wherein the sentence is generated based at least in part on the first metadata and the second metadata;
training the first model using a training data set comprising text data and a structured data representation of the text data;
determining, by a second model, second data indicating that the first natural language representation is a semantically accurate representation of the first data; and
generating an output comprising the first output data, wherein the output is generated based at least in part on the second data.
6. The method of claim 5 , further comprising:
generating, by the first model, second output data comprising a second natural language representation of the first data; and
determining, by the second model, third data indicating that the second natural language representation is a semantically inaccurate representation of the first data, wherein the generating the output comprising the first output data is further based at least in part on the third data.
7. The method of claim 5 , further comprising:
receiving first training data for the second model comprising a first data-text tuple and a first label data associated with the first data-text tuple, wherein the first label data indicates that text of the first data-text tuple is an accurate semantic representation of data of the first data-text tuple;
receiving second training data for the second model comprising a second data-text tuple and a second label data associated with the second data-text tuple, wherein the second label data indicates that text of the second data-text tuple is an inaccurate semantic representation of data of the second data-text tuple; and
updating parameters of the second model based at least in part on the first training data and the second training data.
8. The method of claim 5 , further comprising:
generating, by the first model, second output data comprising a second natural language representation of the first data;
generating, by the first model, a first confidence score associated with the first output data;
generating, by the first model, a second confidence score associated with the second output data; and
selecting the first output data for output further based at least in part on the first confidence score and the second confidence score.
9. The method of claim 5 , further comprising:
generating, by the first model, second output data comprising a second natural language representation of the first data;
generating, by the first model, a first confidence score associated with the first output data;
generating, by the first model, a second confidence score associated with the second output data, the second confidence score being higher than the first confidence score;
determining, by the second model, third data indicating that the second natural language representation is a semantically inaccurate representation of the first data; and
selecting the first output data for output further based at least in part on the third data.
10. The method of claim 5 , wherein the first data comprises triple data including the subject, a predicate, and the object, the method further comprising:
determining, by the first model, third metadata identifying the predicate of the first data; and
wherein the generating, by the first model, the first output data includes omitting the predicate from the first natural language representation based at least in part on the third metadata.
11. The method of claim 5 , further comprising:
generating first training data for a first data-text tuple by duplicating at least one word in the text of the first data-text tuple; and
generating second training data for the first data-text tuple by inserting at least one word from a different data-text tuple into the text of the first data-text tuple.
12. A system, comprising:
at least one processor; and
at least one non-transitory, computer-readable memory storing instructions that, when executed by the at least one processor, are effective to program the at least one processor to:
receive first data comprising a structured representation of linguistic data;
determine first metadata describing a subject of the first data;
determine second metadata describing an object of the first data;
generate, by a first model comprising a computer-implemented neural language model, first output data comprising a first natural language representation of the first data, wherein the first natural language representation comprises a sentence comprising the subject and the object, wherein the sentence is generated based at least in part on the first metadata and the second metadata;
train the first model using a training data set comprising text data and a structured data representation of the text data;
determine, by a second model, second data indicating that the first natural language representation is a semantically accurate representation of the first data; and
generate an output comprising the first output data, wherein the output is generated based at least in part on the second data.
13. The system of claim 12 , wherein the at least one non-transitory, computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
generate, by the first model, second output data comprising a second natural language representation of the first data; and
determine, by the second model, third data indicating that the second natural language representation is a semantically inaccurate representation of the first data, wherein the generating the output comprising the first output data is further based at least in part on the third data.
14. The system of claim 12 , wherein the at least one non-transitory, computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
receive first training data for the second model comprising a first data-text tuple and a first label data associated with the first data-text tuple, wherein the first label data indicates that text of the first data-text tuple is an accurate semantic representation of data of the first data-text tuple;
receive second training data for the second model comprising a second data-text tuple and a second label data associated with the second data-text tuple, wherein the second label data indicates that text of the second data-text tuple is an inaccurate semantic representation of data of the second data-text tuple; and
updating parameters of the second model based at least in part on the first training data and the second training data.
15. The system of claim 12 , wherein the at least one non-transitory, computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
generate, by the first model, second output data comprising a second natural language representation of the first data;
generate, by the first model, a first confidence score associated with the first output data;
generate, by the first model, a second confidence score associated with the second output data; and
select the first output data for output further based at least in part on the first confidence score and the second confidence score.
16. The system of claim 12 , wherein the at least one non-transitory, computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
generate, by the first model, second output data comprising a second natural language representation of the first data;
generate, by the first model, a first confidence score associated with the first output data;
generate, by the first model, a second confidence score associated with the second output data, the second confidence score being higher than the first confidence score;
determine, by the second model, third data indicating that the second natural language representation is a semantically inaccurate representation of the first data; and
select the first output data for output further based at least in part on the third data.
17. The system of claim 12 , wherein the at least one non-transitory, computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
determine, by the first model, third metadata identifying the predicate of the first data; and
wherein the generating, by the first model, the first output data includes omitting the predicate from the first natural language representation based at least in part on the third metadata.
18. The system of claim 12 , wherein the at least one non-transitory, computer-readable memory stores further instructions that, when executed by the at least one processor, are further effective to program the at least one processor to:
generate first training data for a first data-text tuple by duplicating at least one word in the text of the first data-text tuple; and
generate second training data for the first data-text tuple by inserting at least one word from a different data-text tuple into the text of the first data-text tuple.