Reliable gradient-free and likelihood-free prompt tuning
Prompt embedding samples are drawn from a prior distribution and are passed into a pretrained model to receive a corresponding token label prediction for a batch of text data. Prompt embedding samples are accepted from a distribution of a first iteration; the accepted samples satisfy a condition of a distance function between a ground truth label and the corresponding token label prediction being less than a first tolerance. Embeddings are resampled from the accepted prompt embedding samples with probability proportional to weights and the resampled embeddings are perturbed via a perturbation kernel to obtain a new sample. The perturbed resampled embeddings are propagated through the pretrained model, and those that satisfy a condition are projected, where the second tolerance is decayed by one step per iteration. The projected resampled embeddings are concatenated with an embedding of a given input and inferencing is performed.
1 . A method comprising:
drawing, using at least one hardware processor, prompt embedding samples from a prior distribution and passing the prompt embedding samples into a pretrained model to receive a corresponding token label prediction for a batch of text data;
accepting, using the at least one hardware processor, prompt embedding samples from a distribution of a first iteration that satisfy a condition of a distance function between a ground truth label and a corresponding token label prediction being less than a first tolerance;
resampling, using the at least one hardware processor, in a next iteration, embeddings from the accepted prompt embedding samples with probability proportional to weights and perturbing the resampled embeddings via a perturbation kernel to obtain a new sample;
propagating, using the at least one hardware processor, the perturbed resampled embeddings through the pretrained model;
projecting, to a higher dimension than a dimension of the resampled embeddings, using the at least one hardware processor, the resampled embeddings that satisfy a condition of the distance function between the ground truth label and the corresponding token label prediction being less than a second tolerance, where the second tolerance is decayed by one step per iteration;
concatenating the projected resampled embeddings with an embedding of a given input; and
performing inferencing by inputting the concatenated embeddings into the pretrained model.
2 . The method of claim 1 , wherein a number of the prompt embedding samples is designated as S and wherein the second tolerance is decayed by one step per iteration by subtracting an inverse of a total number of training data from the second tolerance.
3 . The method of claim 2 , wherein accuracy is used as the distance function.
4 . The method of claim 2 , wherein a final collection of prompt embedding samples form an approximation to a posterior.
5 . The method of claim 2 , wherein the given input is a text string and the inferencing operation determines a sentiment of the text string.
6 . A computer program product, comprising:
one or more tangible computer-readable storage media and program instructions stored on at least one of the one or more tangible computer-readable storage media, the program instructions executable by a processor, the program instructions comprising:
drawing, using at least one hardware processor, prompt embedding samples from a prior distribution and passing the prompt embedding samples into a pretrained model to receive a corresponding token label prediction for a batch of text data;
accepting, using the at least one hardware processor, prompt embedding samples from a distribution of a first iteration that satisfy a condition of a distance function between a ground truth label and the corresponding token label prediction being less than a first tolerance;
resampling, using the at least one hardware processor in a next iteration, embeddings from the accepted prompt embedding samples with probability proportional to weights and perturbing the resampled embeddings via a perturbation kernel to obtain a new sample;
propagating, using the at least one hardware processor, the perturbed resampled embeddings through the pretrained model;
projecting to a higher dimension than a dimension of the resampled embeddings, using the at least one hardware processor, the resampled embeddings that satisfy a condition of the distance function between the ground truth label and the corresponding token label prediction being less than a second tolerance, where the second tolerance is decayed by one step per iteration;
concatenating the projected resampled embeddings with an embedding of a given input; and
performing inferencing by inputting the concatenated embeddings into the pretrained model.
7 . The computer program product of claim 6 , wherein a number of the prompt embedding samples is designated as S and wherein the second tolerance is decayed by one step per iteration by subtracting an inverse of a total number of training data from the second tolerance.
8 . The computer program product of claim 7 , wherein accuracy is used as the distance function.
9 . The computer program product of claim 7 , wherein a final collection of prompt embedding samples form an approximation to a posterior.
10 . The computer program product of claim 7 , wherein the given input is a text string and the inferencing operation determines a sentiment of the text string.
11 . A system comprising:
a memory; and
at least one processor, coupled to said memory, and operative to perform operations comprising:
drawing, using at least one hardware processor, prompt embedding samples from a prior distribution and passing the prompt embedding samples into a pretrained model to receive a corresponding token label prediction for a batch of text data;
accepting, using the at least one hardware processor, prompt embedding samples from a distribution of a first iteration that satisfy a condition of a distance function between a ground truth label and the corresponding token label prediction being less than a first tolerance;
resampling, using the at least one hardware processor in a next iteration, embeddings from the accepted prompt embedding samples with probability proportional to weights and perturbing the resampled embeddings via a perturbation kernel to obtain a new sample;
propagating, using the at least one hardware processor, the perturbed resampled embeddings through the pretrained model;
projecting to a higher dimension than a dimension of the resampled embeddings, using the at least one hardware processor, the resampled embeddings that satisfy a condition of the distance function between the ground truth label and the corresponding token label prediction being less than a second tolerance, where the second tolerance is decayed by one step per iteration;
concatenating the projected resampled embeddings with an embedding of a given input; and
performing inferencing by inputting the concatenated embeddings into the pretrained model.
12 . The system of claim 11 , wherein a number of the prompt embedding samples is designated as S and wherein the second tolerance is decayed by one step per iteration by subtracting an inverse of a total number of training data from the second tolerance.
13 . The system of claim 12 , wherein accuracy is used as the distance function.
14 . The system of claim 12 , wherein a final collection of prompt embedding samples form an approximation to a posterior.
15 . The system of claim 12 , wherein the given input is a text string and the inferencing operation determines a sentiment of the text string.