System and methods for training task-oriented dialogue (TOD) language models
Embodiments described herein provide methods and systems for training task-oriented dialogue (TOD) language models. In some embodiments, a TOD language model may receive a TOD dataset including a plurality of dialogues and a model input sequence may be generated from the dialogues using a first token prefixed to each user utterance and a second token prefixed to each system response of the dialogues. In some embodiments, the first token or the second token may be randomly replaced with a mask token to generate a masked training sequence and a masked language modeling (MLM) loss may be computed using the masked training sequence. In some embodiments, the TOD language model may be updated based on the MLM loss.
1. A method, comprising:
receiving, at a task-oriented dialogue (TOD) language model, a TOD dataset including a plurality of dialogues, each dialogue of the plurality of dialogues including a plurality of user utterances and a plurality of system responses;
generating a model input sequence by:
prefixing a first token to each user utterance of the plurality of user utterances and a second token to each system response of the plurality of system responses, and
concatenating each of the prefixed user utterances and each of the prefixed system responses;
randomly replacing the first token or the second token from the model input sequence with a mask token to generate a masked training sequence;
inputting the masked training sequence to the TOD language model;
computing a masked language modeling (MLM) loss based on a first output distribution from the TOD language model corresponding to the masked training sequence;
splitting each dialogue from the plurality of dialogues at a respective random turn into a context of the respective dialogue and a response of the respective dialogue;
encoding, by the TOD language model, respective contexts and respective responses corresponding to the plurality of dialogues into a context matrix and a response matrix, respectively;
computing a response contrastive loss (RCL) based on the context matrix and the response matrix generated from the plurality of dialogues; and
updating the TOD language model based on a combination of the MLM loss and the RCL.
2. The method of claim 1 , wherein the context comprises system responses and user utterances prior to a randomly selected dialogue turn, and the response comprises a system response of a next dialogue turn, and the RCL is computed by taking a softmax operation over a product of the content matrix and the response matrix and summing softmax results over the plurality of dialogues.
3. The method of claim 2 , wherein the combination of the MLM loss and the RCL is a weighted sum of the MLM loss and the RCL.
4. The method of claim 1 , wherein the TOD language model is built using a bidirectional encoder representations from transformers (BERT)-based language representation model.
5. The method of claim 1 , further comprising: identifying, using the TOD language model, an intent class of a user utterance of the plurality of user utterances.
6. The method of claim 1 , further comprising: determining, using the TOD language model, a belief state of a dialogue of the plurality of dialogues.
7. The method of claim 1 , further comprising: predicting, using the TOD language model, a dialogue act of a dialogue of the plurality of dialogues.
8. The method of claim 1 , further comprising: selecting, using the TOD language model and for a user utterance from the plurality of user utterances, a system response from the plurality of system responses that is responsive to the user utterance.
9. A system, comprising:
a memory storing instructions for training a task-oriented dialogue (TOD) language model; and
a processor configured to read and execute the instructions from the memory to:
receive, at the TOD language model, a TOD dataset including a plurality of dialogues, each dialogue of the plurality of dialogues including a plurality of user utterances and a plurality of system responses;
generate a model input sequence by:
prefixing a first token to each user utterance of the plurality of user utterances and a second token to each system response of the plurality of system responses, and
concatenating each of the prefixed user utterances and each of the prefixed system responses;
randomly replace the first token or the second token from the model input sequence with a mask token to generate a masked training sequence;
input the masked training sequence to the TOD language model;
compute a masked language modeling (MLM) loss based on a first output distribution from the TOD language model corresponding to the masked training sequence;
split each dialogue from the plurality of dialogues at a respective random turn into a context of the respective dialogue and a response of the respective dialogue;
encode, by the TOD language model, respective contexts and respective responses corresponding to the plurality of dialogues into a context matrix and a response matrix, respectively;
compute a response contrastive loss (RCL) based on the context matrix and the response matrix generated from the plurality of dialogues; and
update the TOD language model based on a combination of the MLM loss and the RCL.
10. The system of claim 9 , wherein the context comprises system responses and user utterances prior to a randomly selected dialogue turn, and the response comprises a system response of a next dialogue turn, and the RCL is computed by taking a softmax operation over a product of the content matrix and the response matrix and summing softmax results over the plurality of dialogues.
11. The system of claim 10 , wherein the combination of the MLM loss and the RCL is a weighted sum of the MLM loss and the RCL.
12. The system of claim 9 , wherein the TOD language model is built using a bidirectional encoder representations from transformers (BERT)-based language representation model.
13. The system of claim 9 , wherein the processor is further configured to read and execute the instructions from the memory to identify, using the TOD language model, an intent class of a user utterance of the plurality of user utterances.
14. The system of claim 9 , wherein the processor is further configured to read and execute the instructions from the memory to determine, using the TOD language model, a belief state of a dialogue of the plurality of dialogues.
15. The system of claim 9 , wherein the processor is further configured to read and execute the instructions from the memory to predict, using the TOD language model, a dialogue act of a dialogue of the plurality of dialogues.
16. The system of claim 9 , wherein the processor is further configured to read and execute the instructions from the memory to select, using the TOD language model and for a user utterance from the plurality of user utterances, a system response from the plurality of system responses that is responsive to the user utterance.
17. A non-transitory computer-readable medium (CRM) having stored thereon machine-readable instructions executable to cause a processor to perform operations comprising:
receiving, at a task-oriented dialogue (TOD) language model, a TOD dataset including a plurality of dialogues, each dialogue of the plurality of dialogues including a plurality of user utterances and a plurality of system responses;
generating a model input sequence by:
(i) prefixing a first token to each user utterance of the plurality of user utterances and a second token to each system response of the plurality of system responses, and
(ii) concatenating each of the prefixed user utterances and each of the prefixed system responses;
randomly replacing the first token or the second token from the model input sequence with a mask token to generate a masked training sequence;
inputting the masked training sequence to the TOD language model;
computing a masked language modeling (MLM) loss based on a first output distribution from the TOD language model corresponding to the masked training sequence;
splitting each dialogue from the plurality of dialogues at a respective random turn into a context of the respective dialogue and a response of the respective dialogue;
encoding, by the TOD language model, respective contexts and respective responses corresponding to the plurality of dialogues into a context matrix and a response matrix, respectively;
computing a response contrastive loss (RCL) based on the context matrix and the response matrix generated from the plurality of dialogues; and
updating the TOD language model based on a combination of the MLM loss and the RCL.
18. The non-transitory CRM of claim 17 , wherein the context comprises system responses and user utterances prior to a randomly selected dialogue turn, and the response comprises a system response of a next dialogue turn, and the RCL is computed by taking a softmax operation over a product of the content matrix and the response matrix and summing softmax results over the plurality of dialogues.
19. The non-transitory CRM of claim 18 , wherein the combination of the MLM loss and the RCL is a weighted sum of the MLM loss and the RCL.
20. The non-transitory CRM of claim 17 , wherein the TOD language model is built using a bidirectional encoder representations from transformers (BERT)-based language representation model.
21. The non-transitory CRM of claim 17 , wherein the operations further comprise identifying, using the TOD language model, an intent class of a user utterance of the plurality of user utterances.
22. The non-transitory CRM of claim 17 , wherein the operations further comprise determining, using the TOD language model, a belief state of a dialogue of the plurality of dialogues.
23. The non-transitory CRM of claim 17 , wherein the operations further comprise predicting, using the TOD language model, a dialogue act of a dialogue of the plurality of dialogues.
24. The non-transitory CRM of claim 17 , wherein the operations further comprise selecting, using the TOD language model and for a user utterance from the plurality of user utterances, a system response from the plurality of system responses that is responsive to the user utterance.