Training natural language processing network based on existing network
Some embodiments provide a method for training a first NLP network based on a previously-trained second NLP network. The method propagates text inputs through the first network to generate a first set of output vectors and the second network to generate a second set of output vectors. Each first-set vector generated based on a text input has a corresponding second-set vector generated based on the same text input and each vector of the first and second sets has a same number of vector components. The method computes a value for a loss function that emphasizes a maximum disparity between components of first-set vectors and corresponding components of second-set vectors. The method trains the first network using the computed loss function value to minimize the maximum disparity between the first-set vector components and corresponding second-set vector components so that the first network produces outputs similar to outputs of the second network.
1 . A method for training a first natural language processing (NLP) network based on a previously-trained second NLP network, the method comprising:
propagating a plurality of text inputs through (1) the first NLP network to generate a first set of output vectors and (2) the previously-trained second NLP network to generate a second set of output vectors, each respective output vector of the first set of output vectors generated based on a respective text input having a corresponding output vector of the second set of output vectors generated based on the respective text input, each respective output vector of the first and second sets of output vectors comprising a same number of respective vector components, each respective output vector for use by a post-processor to select a next word to append to a corresponding text input;
determining, by comparing respective components of a first output vector of the first set of output vectors to corresponding components of a first output vector of the second set of output vectors, a set of per-component disparity values between the first output vector of the first set of output vectors and the first output vector of the second set of output vectors, wherein the first output vector of the first set of output vectors and the first output vector of the second set of output vectors are generated using the same text input;
computing a value for a loss function based on a determination of a maximum disparity among the set of per-component disparity values; and
training the first NLP network using the computed value for the loss function to minimize the maximum disparity so that the first NLP network produces outputs similar to outputs of the previously-trained second NLP network.
2 . The method of claim 1 , wherein, after training, the first NLP network operates to generate output vectors based on input text.
3 . The method of claim 2 , wherein the trained first NLP network:
receives a first text input;
deterministically generates a first output vector; and
provides the first output vector to a non-deterministic token selection function that selects a first token based on the first output vector, wherein the first token is appended to the first text input as a second text input and provided as a second text input to the trained first NLP network.
4 . The method of claim 3 , wherein, until a stopping condition is reached, the trained first NLP network iteratively:
receives a current text input;
deterministically generates an output vector based on the received current text input; and
provides the generated output vector to a non-deterministic token selection function that selects a token based on the first output vector, wherein the token is appended to the current text input as a next text input for the trained first NLP network.
5 . The method of claim 2 , wherein the output vectors represent probability distributions over potential next tokens to append to the input text.
6 . The method of claim 1 , wherein the loss function nests continuously-differentiable functions that estimate a maximum disparity among maximum disparities identified for each output vector of the first set of output vectors and corresponding output vectors of the second set of output vectors.
7 . The method of claim 1 , wherein:
each respective text input comprises a respective plurality of time steps;
each of the first and second NLP networks generates a separate output vector for each time step; and
the loss function nests continuously-differentiable functions that estimate a maximum disparity among the maximum disparities identified for each text input, said maximum disparities each corresponding to a respective time step, and (ii) estimate a maximum disparity among the maximum disparities identified for each output vector of the first set of output vectors and corresponding output vectors of the second set of output vectors.
8 . The method of claim 1 , wherein training the first NLP network to minimize the maximum disparity improves accuracy of the first NLP network for inputs for which the generated first set of output vectors are furthest from expected network outputs.
9 . The method of claim 1 , wherein the loss function uses a log-sum-exponent formulation to estimate the maximum disparity between the components of the first output vector of the first set of output vectors and the corresponding components of the first output vector of the second set of output vectors.
10 . The method of claim 1 , wherein the loss function uses a p-norm function to estimate the maximum disparity between the components of the first output vector of the first set of output vectors and the corresponding components of the first output vector of the second set of output vectors.
11 . The method of claim 1 , wherein the loss function uses a Boltzmann expected energy formulation to estimate the maximum disparity between the components of the first output vector of the first set of output vectors and the corresponding components of the first output vector of the second set of output vectors.
12 . The method of claim 1 , wherein the first NLP network has a first number of transformer decoders and the second NLP network has a second, larger number of transformer decoders.
13 . The method of claim 1 , wherein the first NLP network has a first number of parameters and the second NLP network has a second, larger number of parameters.
14 . A non-transitory machine-readable medium storing a program which when executed by at least one processing unit trains a first natural language processing (NLP) network based on a previously-trained second NLP network, the program comprising sets of instructions for:
propagating a plurality of text inputs through (1) the first NLP network to generate a first set of output vectors and (2) the previously-trained second NLP network to generate a second set of output vectors, each respective output vector of the first set of output vectors generated based on a respective text input having a corresponding output vector of the second set of output vectors generated based on the respective text input, each respective output vector of the first and second sets of output vectors comprising a same number of respective vector components, each respective output vector for use by a post-processor to select a next word to append to a corresponding text input;
determining, by comparing respective components of a first output vector of the first set of output vectors to corresponding components of a first output vector of the second set of output vectors, a set of per-component disparity values between the first output vector of the first set of output vectors and the first output vector of the second set of output vectors, wherein the first output vector of the first set of output vectors and the first output vector of the second set of output vectors are generated using the same text input;
computing a value for a loss function based on an determination of a maximum disparity among the set of per-component disparity values; and
training the first NLP network using the computed value for the loss function to minimize the maximum disparity so that the first NLP network produces outputs similar to outputs of the previously-trained second NLP network.
15 . The non-transitory machine-readable medium of claim 14 , wherein, after training, the first NLP network operates to generate output vectors based on input text.
16 . The non-transitory machine-readable medium of claim 15 , wherein, until a stopping condition is reached, the trained first NLP network iteratively:
receives a current text input;
deterministically generates an output vector based on the received current text input; and
provides the generated output vector to a non-deterministic token selection function that selects a token based on the first output vector, wherein the token is appended to the current text input as a next text input for the trained first NLP network.
17 . The non-transitory machine-readable medium of claim 15 , wherein the output vectors represent probability distributions over potential next tokens to append to the input text.
18 . The non-transitory machine-readable medium of claim 14 , wherein the loss function nests continuously-differentiable functions that estimate a maximum disparity among maximum disparities identified for each output vector of the first set of output vectors and corresponding output vectors of the second set of output vectors.
19 . The non-transitory machine-readable medium of claim 14 , wherein:
each respective text input comprises a respective plurality of time steps;
each of the first and second NLP networks generates a separate output vector for each time step; and
the loss function nests continuously-differentiable functions that estimate a maximum disparity among the maximum disparities identified for each text input, said maximum disparities each corresponding to a respective time step, and (ii) estimate a maximum disparity among the maximum disparities identified for each output vector of the first set of output vectors and corresponding output vectors of the second set of output vectors.