Forecasted diverse natural language generation models
A method, a structure, and a computer system for diverse natural language generation. The exemplary embodiments may include training a data-to-text neural network (D2T NN) and training a text-to-text neural network (T2T NN), wherein the D2T NN and the T2T NN have identical transformer architectures, and wherein the training of the D2T NN and the T2T NN are indexed by time. The exemplary embodiments may further include interleaving weights between the D2T NN and the T2T NN, as well as generating a sentence based on the interleaved D2T NN and T2T NN.
1 . A method for diverse natural language generation, the method comprising:
training a data-to-text neural network (D2T NN) using first training data;
training a text-to-text neural network (T2T NN) using second training data, wherein the D2T NN and the T2T NN have identical transformer architectures, and wherein the training of the D2T NN and the T2T NN are indexed by time;
interleaving weights between the D2T NN and the T2T NN by transferring weights between encoding portions of the D2T NN and the T2T NN, wherein the interleaving minimizes an effect of the first training data and the second training data being collected at different times; and
generating a sentence based on the interleaved D2T NN and T2T NN.
2 . The method of claim 1 , wherein the D2T NN converts one or more statistics into one or more tuples and generates one or more base sentences based on the one or more tuples.
3 . The method of claim 2 , wherein the T2T NN paraphrases the one or more base sentences while adding context.
4 . The method of claim 1 , wherein interleaving the weights between the D2T NN and the T2T NN further comprises:
transferring weights between decoding portions of the D2T NN and the T2T NN.
5 . The method of claim 1 , further comprising:
smoothing the interleaved weights and original weights to a midpoint.
6 . The method of claim 5 , wherein the smoothing is performed via Gibs sampling.
7 . A computer program product for diverse natural language generation, the computer program product comprising:
one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media capable of performing a method, the method comprising:
training a data-to-text neural network (D2T NN) using first training data;
training a text-to-text neural network (T2T NN) using second training data, wherein the D2T NN and the T2T NN have identical transformer architectures, and wherein the training of the D2T NN and the T2T NN are indexed by time;
interleaving weights between the D2T NN and the T2T NN by transferring weights between encoding portions of the D2T NN and the T2T NN, wherein the interleaving minimizes an effect of the first training data and the second training data being collected at different times; and
generating a sentence based on the interleaved D2T NN and T2T NN.
8 . The computer program product of claim 7 , wherein the D2T NN converts one or more statistics into one or more tuples and generates one or more base sentences based on the one or more tuples.
9 . The computer program product of claim 8 , wherein the T2T NN paraphrases the one or more base sentences while adding context.
10 . The computer program product of claim 7 , wherein interleaving the weights between the D2T NN and the T2T NN further comprises:
transferring weights between a decoding portions of the D2T NN and the T2T NN.
11 . The computer program product of claim 7 , further comprising:
smoothing the interleaved weights and original weights to a midpoint.
12 . The computer program product of claim 11 , wherein the smoothing is performed via Gibs sampling.
13 . A computer system for diverse natural language generation, the system comprising:
one or more computer processors, one or more computer-readable storage media, and program instructions stored on the one or more of the computer-readable storage media for execution by at least one of the one or more processors capable of performing a method, the method comprising:
training a data-to-text neural network (D2T NN) using first training data;
training a text-to-text neural network (T2T NN) using second training data, wherein the D2T NN and the T2T NN have identical transformer architectures, and wherein the training of the D2T NN and the T2T NN are indexed by time;
interleaving weights between the D2T NN and the T2T NN by transferring weights between encoding portions of the D2T NN and the T2T NN, wherein the interleaving minimizes an effect of the first training data and the second training data being collected at different times; and
generating a sentence based on the interleaved D2T NN and T2T NN.
14 . The computer system of claim 13 , wherein the D2T NN converts one or more statistics into one or more tuples and generates one or more base sentences based on the one or more tuples.
15 . The computer system of claim 14 , wherein the T2T NN paraphrases the one or more base sentences while adding context.
16 . The computer system of claim 13 , wherein interleaving the weights between the D2T NN and the T2T NN further comprises:
transferring weights between decoding portions of the D2T NN and the T2T NN.
17 . The computer system of claim 13 , further comprising:
smoothing the interleaved weights and original weights to a midpoint.