Multi-turn dialogue response generation with template generation
Machine classifiers in accordance with embodiments of the invention capture long-term temporal dependencies in particular tasks, such as turn-based dialogues. Machine classifiers may be used to help users to perform tasks indicated by the user. When a user utterance is received, natural language processing techniques may be used to understand the user's intent. Templates may be determined based on the user's intent in the generation of responses to solicit information from the user. A variety of persona attributes may be determined for a user. The persona attributes may be determined based on the user's utterances and/or provided as metadata included with the user's utterances. A response persona may be used to generate responses to the user's utterances such that the generated responses match a tone appropriate to the task. A response persona may be used to generate templates to solicit additional information and/or generate responses appropriate to the task.
1. A computer-implemented method comprising:
receiving input data comprising a first user utterance;
generating a first response to the first user utterance by:
determining, based on the first user utterance, a user intent;
determining, based on a conversation history associated with the first user utterance, at least one entity in the first user utterance;
using a machine classifier having a sequence to sequence network architecture:
based on the user intent and the at least one entity:
generating a candidate response; and
obtaining a response template; and
generating, based on the response template and the candidate response, the first response; and
outputting the first response.
2. The computer-implemented method of claim 1 , wherein the machine classifier comprises an encoder and a decoder.
3. The computer-implemented method of claim 1 , further comprising:
updating, based on the first user utterance, the first response, and the at least one entity, the conversation history;
receiving a second user utterance; and
generating, based on the updated conversation history, a second response to the second user utterance.
4. The computer-implemented method of claim 1 , further comprising determining the user intent using natural language processing.
5. The computer-implemented method of claim 1 , wherein the generating the candidate response comprises:
generating an input encoding of the input data;
generating an output sequence comprising:
a start of sequence token; and
one or more output sequence tokens generated by:
providing the input encoding to the machine classifier;
receiving a next output sequence token from the machine classifier; and
appending the next output sequence token to the output sequence until the next output sequence token comprises an end of sequence token; and
generating the candidate response based on the output sequence.
6. The computer-implemented method of claim 1 , wherein the obtaining the response template comprises generating, by the machine classifier, the response template based on the first user utterance, the at least one entity, and the user intent, and wherein the response template and the candidate response are generated in parallel.
7. The computer-implemented method of claim 1 , further comprising training the machine classifier, using a plurality of training sequences, wherein each training sequence comprises an encoder sequence and a decoder sequence.
8. The computer-implemented method of claim 7 , wherein the training the machine classifier comprises:
generating, for each training sequence of the plurality of training sequences, an encoding of the encoder sequence of the training sequence and the decoder sequence of the training sequence; and
for each encoding:
padding the encoder sequence of the encoding with an informative padding;
prepending a start of sequence token to the encoder sequence of the encoding;
appending an end of sequence token to the decoder sequence of the encoding;
training, using the encoder sequence of the encoding, an encoder of the machine classifier; and
training, using the decoder sequence of the encoding, a decoder of the machine classifier.
9. The computer-implemented method of claim 7 , wherein the training the machine classifier comprises:
updating an attention weight associated with at least one token in an encoder sequence; and
updating an attention weight associated with at least one token in a decoder sequence.
10. The computer-implemented method of claim 1 , wherein:
the input data comprises a multi-turn dialog; and
the machine classifier is trained to generate responses regarding multi-turn dialogs.
11. A device comprising:
a processor; and
a memory storing computer-readable instructions that, when executed by the processor, cause the device to:
receive input data comprising multi-turn dialog, wherein the multi-turn dialog comprises a conversation history and a first user utterance;
generate a first response to the first user utterance by:
determine, based on the first user utterance and using natural language processing, a user intent;
determine, based on the conversation history and the first user utterance, at least one entity in the first user utterance; and
using a machine classifier having a sequence to sequence network architecture:
generate, based on the user intent and the at least one entity, a response template and a candidate response; and
generate, based on the response template and the candidate response, the first response; and
output the first response.
12. The device of claim 11 , wherein the machine classifier comprises an encoder and a decoder.
13. The device of claim 11 , wherein the computer-readable instructions, when executed by the processor, further cause the device to:
update, based on the first user utterance, the first response, and the at least one entity, the conversation history;
receive a second user utterance; and
generate, based on the updated conversation history, a second response to the second user utterance.
14. The device of claim 11 , wherein the computer-readable instructions, when executed by the processor, cause the device to generate the candidate response by causing the device to:
generate an input encoding of the input data;
generate an output sequence comprising:
a start of sequence token; and
one or more output sequence tokens generated by causing the device to:
provide the input encoding to the machine classifier:
receive a next output sequence token from the machine classifier; and
append the next output sequence token to the output sequence until the next output sequence token comprises an end of sequence token; and
generate the candidate response based on the output sequence.
15. The device of claim 11 , wherein the computer-readable instructions, when executed by the processor, cause the device to generate the response template further based on the first user utterance, wherein the response template and the candidate response are generated in parallel.
16. The device of claim 11 , wherein the computer-readable instructions, when executed by the processor, further cause the device to train the machine classifier, using a plurality of training sequences, wherein each training sequence comprises an encoder sequence and a decoder sequence.
17. The device of claim 16 , wherein the computer-readable instructions, when executed by the processor, cause the device to train the machine classifier by causing the device to:
generate, for each training sequence of the plurality of training sequences, an encoding of the encoder sequence of the training sequence and the decoder sequence of the training sequence; and
for each encoding:
pad the encoder sequence of the encoding with an informative padding;
prepend a start of sequence token to the encoder sequence of the encoding;
append an end of sequence token to the decoder sequence of the encoding;
train, using the encoder sequence of the encoding, an encoder of the machine classifier; and
train, using the decoder sequence of the encoding, a decoder of the machine classifier.
18. The device of claim 16 , wherein the computer-readable instructions, when executed by the processor, cause the device to train the machine classifier by causing the device to:
update an attention weight associated with at least one token in an encoder sequence; and
update an attention weight associated with at least one token in a decoder sequence.
19. A non-transitory, computer-readable medium storing instructions that, when executed, cause:
training a machine classifier having a sequence to sequence network architecture;
receiving input data comprising a conversation history and a first user utterance;
generating a first response to the first user utterance by:
determining, based on the first user utterance, a user intent;
determining, based on the conversation history and the first user utterance, at least one entity in the first user utterance; and
using the machine classifier:
generating, based on the user intent and the at least one entity, a response template and a candidate response; and
generating, based on the response template and the candidate response, the first response; and
outputting the first response.
20. The non-transitory, computer-readable medium of claim 19 , wherein the machine classifier comprises an encoder and a decoder.
21. The non-transitory, computer-readable medium of claim 19 , wherein the instructions, when executed, further cause:
updating, based on the first user utterance, the first response, and the at least one entity, the conversation history;
receiving a second user utterance; and
generating, based on the updated conversation history, a second response to the second user utterance.
22. The non-transitory, computer-readable medium of claim 19 , wherein the instructions, when executed, further cause generating the candidate response by:
generating an input encoding of the input data;
generating an output sequence comprising:
a start of output sequence token; and
one or more output sequence tokens generated by:
providing the input encoding to the machine classifier;
receiving a next output sequence token from the machine classifier; and
appending the next output sequence token to the output sequence until the next output sequence token comprises an end of output sequence token; and
generating the candidate response based on the output sequence.
23. The non-transitory, computer-readable medium of claim 19 , wherein training the machine classifier comprises:
training the machine classifier, using a plurality of training sequences, wherein each training sequence comprises an encoder sequence and a decoder sequence;
generating, for each training sequence of the plurality of training sequences, an encoding of the encoder sequence of the training sequence and the decoder sequence of the training sequence; and
for each encoding:
padding the encoder sequence of the encoding with an informative padding;
prepending a start of sequence token to the encoder sequence of the encoding;
appending an end of sequence token to the decoder sequence of the encoding;
training, using the encoder sequence of the encoding, an encoder of the machine classifier; and
training, using the decoder sequence of the encoding, a decoder of the machine classifier.
24. The non-transitory, computer-readable medium of claim 23 , wherein the instructions, when executed, cause training the machine classifier by:
updating an attention weight associated with at least one token in an encoder sequence; and
updating an attention weight associated with at least one token in a decoder sequence.
25. The non-transitory, computer-readable medium of claim 19 , wherein:
the input data comprises a multi-turn dialog; and
the machine classifier is trained to generate responses regarding multi-turn dialogs.