Specialized token prediction by a large language model to prompt external intervention
Certain aspects provide methods for training and using large language models (LLMs) to predict specialized tokens for prompting human intervention. A method includes obtaining a plurality of training data instances, each including: a training input including a first timestamp and a prompt and/or an intermediate response to the prompt, and a training output including a second timestamp and a response. The method includes annotating the training output of one or more training data instances to include an action token indicating that external intervention is required, wherein the annotation, for each training data instance, is based on: a time difference between the second timestamp and the first timestamp, a number of words included in the response; or at least one trigger word included in the response. The method also includes training the LLM on the training data instances to predict when external intervention is required and accordingly generate the action token.
1 . A method for training a large language model (LLM) to predict action tokens, comprising:
obtaining a plurality of training data instances, wherein each of the plurality of training data instances comprises:
a training input comprising a first timestamp and at least one of a prompt or an intermediate response to the prompt; and
a training output comprising a second timestamp and a response to the prompt or the intermediate response;
annotating the training output of a respective training data instance of the plurality of training data instances to include an action token indicating that external intervention is required between the first timestamp and the second timestamp of the respective training data instance, wherein the annotating is based on at least one of:
a time difference between the second timestamp and the first timestamp of the respective training data instance;
a number of words included in the response associated with the training output of the respective training data instance; or
at least one trigger word included in the response associated with the training output of the respective training data instance; and
training the LLM on the plurality of training data instances to predict when external intervention is required and accordingly generate the action token to invoke the external intervention.
2 . The method of claim 1 , wherein annotating the training output of the respective training data instance to include the action token based on the time difference between the second timestamp and the first timestamp and the number of words included in the response associated with the training output of the respective training data instance comprises:
calculating the time difference between the second timestamp and the first timestamp;
determining the number of words included in the response associated with the training output;
calculating a speed metric as a ratio of the number of words included in the response associated with the training output to the time difference;
determining the speed metric is greater than a threshold; and
annotating the training output of the respective training data instance to include the action token based on the speed metric being greater than the threshold.
3 . The method of claim 1 , wherein annotating the training output of the respective training data instance to include the action token based on the at least one trigger word included in the response associated with the training output of the respective training data instance comprises:
determining at least one word included in the response is included in a list of known trigger words; and
annotating the training output of the respective training data instance to include the action token based on the determination.
4 . The method of claim 1 , further comprising training the LLM to pause response generation after generating the action token.
5 . The method of claim 1 , wherein the response to the prompt or the intermediate response included in the training output of each of the plurality of training data instances comprises a next-in-time response to the prompt or the intermediate response.
6 . An apparatus, comprising:
one or more memories comprising processor-executable instructions;
and one or more processors configured to execute the processor-executable instructions and cause the apparatus to:
obtain a plurality of training data instances, wherein each of the plurality of training data instances comprises:
a training input comprising a first timestamp and at least one of a prompt or an intermediate response to the prompt; and
a training output comprising a second timestamp and a response to the prompt or the intermediate response;
annotate the training output of a respective training data instance of the plurality of training data instances to include an action token indicating that external intervention is required between the first timestamp and the second timestamp of the respective training data instance, wherein the annotation is based on at least one of:
a time difference between the second timestamp and the first timestamp of the respective training data instance;
a number of words included in the response associated with the training output of the respective training data instance; or
at least one trigger word included in the response associated with the training output of the respective training data instance; and
train a large language model (LLM) on the plurality of training data instances to predict when external intervention is required and accordingly generate the action token to invoke the external intervention.
7 . The apparatus of claim 6 , wherein to annotate the training output of the respective training data instance to include the action token based on the time difference between the second timestamp and the first timestamp and the number of words included in the response associated with the training output of the respective training data instance, the one or more processors are configured to execute the processor-executable instructions and cause the apparatus to:
calculate the time difference between the second timestamp and the first timestamp;
determine the number of words included in the response associated with the training output;
calculate a speed metric as a ratio of the number of words included in the response associated with the training output to the time difference;
determine the speed metric is greater than a threshold; and
annotate the training output of the respective training data instance to include the action token based on the speed metric being greater than the threshold.
8 . The apparatus of claim 6 , wherein to annotate the training output of the respective training data instance to include the action token based on the at least one trigger word included in the response associated with the training output of the respective training data instance, the one or more processors are configured to execute the processor-executable instructions and cause the apparatus to:
determine at least one word included in the response is included in a list of known trigger words; and
annotate the training output of the respective training data instance to include the action token based on the determination.
9 . The apparatus of claim 6 , wherein the one or more processors are configured to execute the processor-executable instructions and further cause the apparatus to train the LLM to pause response generation after generating the action token.
10 . The apparatus of claim 6 , wherein the response to the prompt or the intermediate response included in the training output of each of the plurality of training data instances comprises a next-in-time response to the prompt or the intermediate response.