Systems and methods for shared latent space prompt tuning
Embodiments described herein provide a prompt-based transfer learning method that employs shared latent space prompt tuning). Specifically, a shared latent space is assumed, among all source and target tasks, where each vector in the space captures a basis skill to do a particular task. Given an instance (from either a source task or a target task), it is first encoded into an instance representation vector and then queries the latent space, which yields a skill vector for this instance. This vector modulates a frozen model, via soft prompts which are a simple prompt transformation (the prompt generator in FIG. 3 ) of the basis skill vector, to generate an answer for the instance. The latent space and prompt transformation are learned end-to-end in upstream pre-training on source tasks.
1. A method of shared prompt-tuning in language models, the method comprising:
receiving, via a data interface, a training dataset of instances;
encoding, by an instance encoder, a first instance from the training dataset into a first embedding in a latent space;
querying a skill latent space on the first embedding to obtain a first skill vector;
transforming, by a prompt generator, the first skill vector into a set of prompt tokens;
generating, by a pretrained language model, an answer in response to an input of the first instance prepended with the prompt tokens;
computing a first training objective as a cross-entropy loss between a conditional distribution of the generated answer conditioned on the prompt generator and the first instance and a first label corresponding to the first instance from the training dataset;
computing a second training objective based on a difference between a stop gradient operator over the first embedding and the first skill vector;
computing a third training objective as a weighted sum of the first training objective and the second training objective; and
updating the prompt generator based on the third training objective via backpropagation while the pre-trained language model and the instance encoder are frozen.
2. The method of claim 1 , wherein the training dataset includes instances and corresponding labels, wherein each pair of instance and corresponding label corresponds to a source natural language processing (NLP) task or a target NLP task.
3. The method of claim 1 , wherein the querying the skill latent space includes:
retrieving a tunable set of skill vectors in the latent space; and
querying the tunable set of skill vectors on the first embedding by determining a nearest neighbor to the first embedding as a first skill vector for the first instance.
4. The method of claim 1 , further comprising:
updating both the prompt generator and the tunable set of skill vectors based on the third training objective via backpropagation while the pre-trained language model and the instance encoder are frozen,
when the first instance is selected from a mixture of source NLP tasks.
5. The method of claim 1 , further comprising:
updating only the prompt generator based on the first training objective via backpropagation while the tunable set of skill vectors, the pre-trained language model and the instance encoder are frozen,
when the first instance is selected only for the target NLP task.
6. The method of claim 1 , wherein the prompt generator is a light-weight multilayer perceptron (MLP).
7. The method of claim 1 , wherein the set of prompt tokens are generated by at least a tangent transformation of a linear transformation of the first skill vector.
8. The method of claim 1 , further comprising:
pre-computing, by the instance encoder, embeddings for instances in the training dataset; and
retrieving the first embedding for the first instance at training stage.
9. A system of shared prompt-tuning in language models, the system comprising:
a data interface that receives a training dataset of instances;
a memory storing an instance encoder, a skill vector generator, a prompt generator and a pretrained language model, and a plurality of processor-executable instructions; and
one or more processors reading and executing the plurality of processor-executable instructions to perform:
encoding, by the instance encoder, a first instance from the training dataset into a first embedding in a latent space;
querying a skill latent space on the first embedding to obtain a first skill vector;
transforming, by the prompt generator, the first skill vector into a set of prompt tokens;
generating, by the pretrained language model, an answer in response to an input of the first instance prepended with the prompt tokens;
computing a first training objective as a cross-entropy loss between a conditional distribution of the generated answer conditioned on the prompt generator and the first instance and a first label corresponding to the first instance from the training dataset;
computing a second training objective based on a difference between a stop gradient operator over the first embedding and the first skill vector;
computing a third training objective as a weighted sum of the first training objective and the second training objective; and
updating the prompt generator based on the third training objective via backpropagation while the pre-trained language model and the instance encoder are frozen.
10. The system of claim 9 , wherein the training dataset includes instances and corresponding labels, wherein each pair of instance and corresponding label corresponds to a source natural language processing (NLP) task or a target NLP task.
11. The system of claim 9 , wherein an operation of querying the skill latent space includes:
retrieving a tunable set of skill vectors in the latent space; and
querying the tunable set of skill vectors on the first embedding by determining a nearest neighbor to the first embedding as a first skill vector for the first instance.
12. The system of claim 9 , wherein the operations further comprise:
updating both the prompt generator and the tunable set of skill vectors based on the third training objective via backpropagation while the pre-trained language model and the instance encoder are frozen,
when the first instance is selected from a mixture of source NLP tasks.
13. The system of claim 9 , wherein the operations further comprise:
updating only the prompt generator based on the first training objective via backpropagation while the tunable set of skill vectors, the pre-trained language model and the instance encoder are frozen,
when the first instance is selected only for the target NLP task.
14. The system of claim 9 , wherein the prompt generator is a light-weight multilayer perceptron (MLP).
15. The system of claim 9 , wherein the set of prompt tokens are generated by at least a tangent transformation of a linear transformation of the first skill vector.
16. A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for shared prompt-tuning in language models, the instructions being executed by one or more processors to perform operations comprising:
receiving, via a data interface, a training dataset of instances;
encoding, by an instance encoder, a first instance from the training dataset into a first embedding in a latent space;
querying a skill latent space on the first embedding to obtain a first skill vector;
transforming, by a prompt generator, the first skill vector into a set of prompt tokens;
generating, by a pretrained language model, an answer in response to an input of the first instance prepended with the prompt tokens;
computing a first training objective as a cross-entropy loss between a conditional distribution of the generated answer conditioned on the prompt generator and the first instance and a first label corresponding to the first instance from the training dataset;
computing a second training objective based on a difference between a stop gradient operator over the first embedding and the first skill vector;
computing a third training objective as a weighted sum of the first training objective and the second training objective; and
updating the prompt generator based on the third training objective via backpropagation while the pre-trained language model and the instance encoder are frozen.