Dialog generation method, apparatus, and electronic device
A dialog generation method includes: training a sequence to sequence (seq2seq)-based dialog model using a loss function including topic range constraint information; and generating a dialog using the trained dialog model. With the dialog generation method, topic range constraint information is introduced in the process of dialog model training using a loss function including the topic range constraint information, thus helping to prevent the trained model from producing low-quality meaningless replies.
1. A dialog generation method, comprising:
training a topic model using a dialog training corpus to obtain multiple first topic vectors, each of the multiple first topic vectors representing a topic;
training a sequence to sequence (seq2seq)-based dialog model based on the trained topic model and a loss function comprising topic range constraint information; and
generating a dialog using the trained dialog model,
wherein the training the seq2seq-based dialog model comprises:
inputting a training sequence into the seq2seq-based dialog model for processing to obtain various hidden-state output data output during the processing and an output sequence obtained after the processing;
generating a second topic vector corresponding to the output sequence according to the various hidden-state output data;
inputting the output sequence into the trained topic model for processing to obtain a first topic vector corresponding to the output sequence among the multiple first topic vectors;
calculating a loss using the loss function comprising topic range constraint information according to the first and second topic vectors corresponding to the output sequence; and
adjusting the dialog model according to the loss.
2. The method of claim 1 , wherein the generating a second topic vector corresponding to the output sequence according to the various hidden-state output data comprises:
pooling the various hidden-state output data; and
inputting data obtained after the pooling into a multi-layer perceptron for processing to generate the second topic vector corresponding to the output sequence.
3. The method of claim 1 , wherein the calculating a loss using the loss function comprising topic range constraint information according to the first and second topic vectors corresponding to the output sequence comprises:
calculating a first similarity between the second topic vector corresponding to the output sequence and the first topic vector corresponding to the output sequence;
calculating a second similarity between the second topic vector corresponding to the output sequence and an irrelevant topic vector, wherein the irrelevant topic vector is a vector in the multiple first topic vectors other than the first topic vector corresponding to the output sequence; and
substituting the first similarity and the second similarity into the loss function comprising topic range constraint information to calculate the loss, wherein the topic range constraint information is a formula comprising parameters corresponding to the first similarity and the second similarity.
4. A dialog generation device, comprising:
a processor; and
a memory for storing instructions executable by the processor;
wherein the processor is configured to:
train a topic model using a dialog training corpus to obtain multiple first topic vectors, each of the multiple first topic vectors representing a topic;
train a sequence to sequence (seq2seq)-based dialog model based on the trained topic model and a loss function comprising topic range constraint information; and
generate a dialog using the trained dialog model,
wherein in training the seq2seq-based dialog model, the processor is further configured to:
input a training sequence into the seq2seq-based dialog model for processing to obtain various hidden-state output data output during the processing and an output sequence obtained after the processing;
generate a second topic vector corresponding to the output sequence according to the various hidden-state output data;
input the output sequence into the trained topic model for processing to obtain a first topic vector corresponding to the output sequence among the multiple first topic vectors;
calculate a loss using the loss function comprising topic range constraint information according to the first and second topic vectors corresponding to the output sequence; and
adjust the dialog model according to the loss.
5. The device of claim 4 , wherein in generating a second topic vector corresponding to the output sequence according to the various hidden-state output data, the processor is further configured to:
pool the various hidden-state output data; and
input data obtained after the pooling into a multi-layer perceptron for processing to generate the second topic vector corresponding to the output sequence.
6. The device of claim 4 , wherein in calculating the loss using the loss function comprising topic range constraint information according to the first and second topic vectors corresponding to the output sequence, the processor is further configured to:
calculate a first similarity between the second topic vector corresponding to the output sequence and the first topic vector corresponding to the output sequence;
calculate a second similarity between the second topic vector corresponding to the output sequence and an irrelevant topic vector, wherein the irrelevant topic vector is a vector in the multiple first topic vectors other than the first topic vector corresponding to the output sequence; and
substitute the first similarity and the second similarity into the loss function comprising topic range constraint information to calculate the loss, wherein the topic range constraint information is a formula comprising parameters corresponding to the first similarity and the second similarity.
7. A non-transitory computer-readable storage medium having stored therein instructions that, when executed by a processor of a device, cause the device to perform a dialog generation method, the method comprising:
training a topic model using a dialog training corpus to obtain multiple first topic vectors, each of the multiple first topic vectors representing a topic;
training a sequence to sequence (seq2seq)-based dialog model based on the trained topic model and a loss function comprising topic range constraint information; and
generating a dialog using the trained dialog model,
wherein the training the seq2seq-based dialog model comprises:
inputting a training sequence into the seq2seq-based dialog model for processing to obtain various hidden-state output data output during the processing and an output sequence obtained after the processing;
generating a second topic vector corresponding to the output sequence according to the various hidden-state output data;
inputting the output sequence into the trained topic model for processing to obtain a first topic vector corresponding to the output sequence among the multiple first topic vectors;
calculating a loss using the loss function comprising topic range constraint information according to the first and second topic vectors corresponding to the output sequence; and
adjusting the dialog model according to the loss.
8. The non-transitory computer-readable storage medium of claim 7 , wherein the generating a second topic vector corresponding to the output sequence according to the various hidden-state output data comprises:
pooling the various hidden-state output data; and
inputting data obtained after the pooling into a multi-layer perceptron for processing to generate the second topic vector corresponding to the output sequence.
9. The non-transitory computer-readable storage medium of claim 7 , wherein the calculating a loss using the loss function comprising topic range constraint information according to the first and second topic vectors corresponding to the output sequence comprises:
calculating a first similarity between the second topic vector corresponding to the output sequence and the first topic vector corresponding to the output sequence;
calculating a second similarity between the second topic vector corresponding to the output sequence and an irrelevant topic vector, wherein the irrelevant topic vector is a vector in the multiple first topic vectors other than the first topic vector corresponding to the output sequence; and
substituting the first similarity and the second similarity into the loss function comprising topic range constraint information to calculate the loss, wherein the topic range constraint information is a formula comprising parameters corresponding to the first similarity and the second similarity.