Machine-learned language models which generate intermediate textual analysis in service of contextual text generation
The present disclosure is directed to systems and methods that include and/or leverage one or more machine-learned language models that generate intermediate textual analysis (e.g., including usage of structural tools such as APIs) in service of contextual text generation. For example, a computing system can obtain a contextual text string that includes one or more contextual text tokens. The computing system can process the contextual text string with the machine-learned language model to generate one or more intermediate text strings that include one or more intermediate text tokens. The computing system can process the one or more intermediate text strings with the machine-learned language model to generate an output text string comprising one or more output text tokens. The one or more intermediate text strings can include textual analysis of the contextual text string that supports the output text string.
1 . A computer-implemented method for training a machine-learned model to generate intermediate tokens for processing by an attention mechanism of the machine-learned model to improve subsequent outputs of the machine-learned model, the method comprising:
obtaining, by a computing system comprising one or more computing devices, a training example for training the machine-learned model to perform a contextual content generation task, wherein the training example comprises:
an initial portion comprising a context sequence;
an intermediate portion comprising an intermediate sequence; and
a response portion comprising a response sequence;
processing, by the computing system, the initial portion using the machine-learned model, wherein the machine-learned model uses the attention mechanism to perform attention over the initial portion;
generating, by the computing system and based on performing attention over the initial portion with the attention mechanism, one or more intermediate token predictions;
determining, by the computing system and using the intermediate sequence, one or more first loss values for the one or more intermediate token predictions;
generating, by the computing system and based on performing attention over the initial portion and the intermediate portion with the attention mechanism, one or more response token predictions;
determining, by the computing system and using the response sequence, one or more second loss values for the one or more response token predictions; and
training, by the computing system, the machine-learned model based on the one or more first loss values and the one or more second loss values.
2 . The method of claim 1 , wherein the one or more intermediate token predictions correspond to one or more tool tokens.
3 . The method of claim 1 , wherein the one or more intermediate token predictions correspond to step-by-step logic associated with the initial portion.
4 . The method of claim 1 , wherein the initial portion is marked with a first tag that indicates an entity associated with the context sequence.
5 . The method of claim 4 , wherein the first tag indicates that the context sequence is associated with a user.
6 . The method of claim 5 , wherein the first tag comprises the textual string, “user”.
7 . The method of claim 2 , wherein the intermediate portion is marked with one or more second tags that indicate an input to a tool associated with the one or more tool tokens.
8 . The method of claim 7 , wherein the one or more second tags comprise:
an input tag that indicates an input to the tool; and
an output tag that indicates an output of the tool.
9 . The method of claim 1 , wherein the response portion is marked with a third tag that indicates an entity associated with the response sequence.
10 . The method of claim 9 , wherein the third tag indicates that the response sequence is associated with an agent or assistant.
11 . The method of claim 10 , wherein the third tag comprises the textual string, “assistant”.
12 . The method of claim 1 , wherein the machine-learned model is a left-to-right language model configured to predict a next token based on one or more preceding tokens.
13 . The method of claim 1 , wherein the machine-learned model is implemented by a dialog agent configured to conduct a dialogue responsive to user inputs.
14 . The method of claim 13 , wherein the training example comprises a dialogue between a user and an assistant, wherein assistant portions of the dialogue are generated by the machine-learned model.
15 . A computing system for training a machine-learned model to generate intermediate tokens for processing by an attention mechanism of the machine-learned model to improve subsequent outputs of the machine-learned model, the computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining a training example for training the machine-learned model to perform a contextual content generation task, wherein the training example comprises:
an initial portion comprising a context sequence;
an intermediate portion comprising an intermediate sequence; and
a response portion comprising a response sequence;
processing the initial portion using the machine-learned model, wherein the machine-learned model uses the attention mechanism to perform attention over the initial portion;
generating, based on performing attention over the initial portion with the attention mechanism, one or more intermediate token predictions;
determining, using the intermediate sequence, one or more first loss values for the one or more intermediate token predictions;
generating, based on performing attention over the initial portion and the intermediate portion with the attention mechanism, one or more response token predictions;
determining, using the response sequence, one or more second loss values for the one or more response token predictions; and
training the machine-learned model based on the one or more first loss values and the one or more second loss values.
16 . The computing system of claim 15 , wherein the one or more intermediate token predictions correspond to one or more tool tokens.
17 . The computing system of claim 15 , wherein the one or more intermediate token predictions correspond to step-by-step logic associated with the initial portion.
18 . The computing system of claim 15 , wherein:
the initial portion is marked with a first tag that indicates an entity associated with the context sequence, wherein the first tag indicates that the context sequence is associated with a user; and
the response portion is marked with a third tag that indicates an entity associated with the response sequence, wherein the third tag indicates that the response sequence is associated with an agent or assistant.
19 . The computing system of claim 18 , wherein the first tag comprises the textual string, “user,” and the third tag comprises the textual string, “assistant”.
20 . One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations for training a machine-learned model to generate intermediate tokens for processing by an attention mechanism of the machine-learned model to improve subsequent outputs of the machine-learned model, the operations comprising:
obtaining a training example for training the machine-learned model to perform a contextual content generation task, wherein the training example comprises:
an initial portion comprising a context sequence;
an intermediate portion comprising an intermediate sequence; and
a response portion comprising a response sequence;
processing the initial portion using the machine-learned model, wherein the machine-learned model uses the attention mechanism to perform attention over the initial portion;
generating, based on performing attention over the initial portion with the attention mechanism, one or more intermediate token predictions;
determining, using the intermediate sequence, one or more first loss values for the one or more intermediate token predictions;
generating, based on performing attention over the initial portion and the intermediate portion with the attention mechanism, one or more response token predictions;
determining, using the response sequence, one or more second loss values for the one or more response token predictions; and
training the machine-learned model based on the one or more first loss values and the one or more second loss values.