Granted Patent B2

US 12,242,810 · App. 17/631,666 · Granted Mar 4, 2025

Inventors: Pingping Lin (Redmond, WA); Ruihua Song (Beijing, CN); Lei Ding (Beijing, CN); Yue Liu (Redmond, WA); Min Zeng (Redmond, WA)

Assignee: Microsoft Technology Licensing, LLC

G06F40/35G06F16/3329G06F40/253H04L51/02

Abstract

Claims (82)

1. A method for performing context completion to messages in a session, comprising:

obtaining a first message;

detecting a context ellipsis in the first message;

determining whether the first message is retained in a current domain of the session;

in response to determining that the first message is retained in the current domain, selecting, in the current domain, a complementary text for recovering the first message context ellipsis using neural network-based ranking, wherein the selecting includes:

calculating a matrix between the complementary text and the first message where the complementary text includes n values and the first message includes m values in the matrix;

extracting a feature from the matrix using a convolutional layer that is coupled with a pooling layer;

outputting a result based on the matrix and the feature to a connection layer; and

generating a ranking using the result where the selecting is based on the ranking;

generating a completed message based on the first message and the complementary text;

obtaining a second message;

detecting a context ellipsis in the second message;

determining the second message context ellipsis is a predetermined dialogue act message; and

providing a response to the second message, the response being provided based on a response strategy corresponding to the predetermined dialogue act message.

2. The method of claim 1 , wherein the detecting that there exists context ellipsis in the first message comprises:

extracting a topic and/or predicate from the first message; and

determining that there exists topic ellipsis or predicate ellipsis in the first message based on a result of the extracting the topic.

3. The method of claim 2 , further comprising:

adding the extracted topic and/or predicate into a domain topic flow and/or domain predicate flow of the current domain respectively.

4. The method of claim 2 , further comprising:

determining session level information associated with the extracted topic and/or predicate; and

adding the session level information into a domain topic flow and/or domain predicate flow of the current domain respectively.

5. The method of claim 2 , wherein, the extracting the topic is performed based at least on a pre-established knowledge graph.

6. The method of claim 2 , wherein the determining whether the first message is retained in the current domain of the session comprises:

in response to determining that there exists the topic ellipsis in the first message, determining whether the first message is retained in the current domain through a domain retaining classifier.

7. The method of claim 6 , wherein,

the domain retaining classifier is based on a recurrent neural network (RNN) model, and

the domain retaining classifier is obtained, through knowledge distillation, from a classifier which is based on a bidirectional encoder representations from transformers (BERT) model.

8. The method of claim 2 , wherein the determining whether the first message is retained in the current domain of the session comprises:

in response to determining that there exists the predicate ellipsis in the first message, determining whether the first message is retained in the current domain based on a topic included in the first message.

9. The method of claim 3 , wherein the selecting a complementary text comprises:

in response to determining that there exists the topic ellipsis in the first message, selecting the complementary text from the domain topic flow; or

in response to determining that there exists the predicate ellipsis in the first message, selecting the complementary text from the domain predicate flow.

10. The method of claim 3 , wherein the selecting a complementary text comprises:

selecting the complementary text from a plurality of candidate texts in the domain topic flow and/or domain predicate flow, through at least one of convolutional neural network (CNN)-based ranking and regression-based ranking.

11. The method of claim 10 , wherein,

the CNN-based ranking is performed according to at least one of a following information of a candidate text:

text similarity between the candidate text and the first message,

pointwise mutual information (PMI) between the candidate text and the first message,

PMI between the candidate text and a synonymous representation of the first message, and

PMI between a synonymous representation of the candidate text and the first message; and

the regression-based ranking is performed according to at least one of a following features of a candidate text:

a frequency that the candidate text has occurred in the session,

whether the candidate text occurred in a last message,

whether the candidate text occurred in a last response,

a number of turns having passed since the candidate text occurred last time in the session,

PMI between the candidate text and the first message, and

text similarity between the candidate text and the first message.

12. The method of claim 1 , wherein the generating a completed message comprises:

generating at least one candidate completed message through placing the complementary text at different positions of the first message; and

selecting the completed message from the at least one candidate completed message through a language model.

13. The method of claim 1 , further comprising:

in response to determining that the first message is not retained in the current domain, determining a domain corresponding to the first message;

selecting, in the domain corresponding to the first message, a complementary text for recovering the context ellipsis; and

generating a completed message based on the first message and the complementary text.

14. An apparatus for performing context completion to messages in a session, comprising:

a message obtaining module, for obtaining a first message;

a context ellipsis detecting module, for detecting a context ellipsis in the first message;

a domain retaining determining module, for determining whether the first message is retained in a current domain of the session;

a complementary text selecting module, for in response to determining that the first message is retained in the current domain, selecting, in the current domain, a complementary text for recovering the context ellipsis using neural network-based ranking, wherein the selecting includes:

calculating a matrix between the complementary text and the first message where the complementary text includes n values and the first message includes m values in the matrix;

extracting a feature from the matrix using a convolutional layer that is coupled with a pooling layer;

outputting a result based on the matrix and the feature to a connection layer; and

generating a ranking using the result where the selecting is based on the ranking;

a completed message generating module, for generating a completed message based on the first message and the complementary text;

a module for obtaining a second message;

a module for detecting a context ellipsis in the second message;

a module for determining the second message context ellipsis is a predetermined dialogue act message; and

a module for providing a response to the second message, the response being provided based on a response strategy corresponding to the predetermined dialogue act message.

15. An apparatus for performing context completion to messages in a session, comprising:

at least one processor; and

a memory storing computer-executable instructions that, when executed, cause the at least one processor to:

obtain a first message,

detect a context ellipsis in the first message,

determine whether the first message is retained in a current domain of the session,

in response to determining that the first message is retained in the current domain, select, in the current domain, a complementary text for recovering the first context ellipsis using neural network-based ranking, wherein the selecting includes: calculating a matrix between the complementary text and the first message where the complementary text includes n values and the first message includes m values in the matrix; extracting a feature from the matrix using a convolutional layer that is coupled with a pooling layer; outputting a result based on the matrix and the feature to a connection layer; and generating a ranking using the result where the selecting is based on the ranking,

generate a completed message based on the first message and the complementary text,

obtain a second message,

detect a context ellipsis in the second message,

determine the second message context ellipsis is a predetermined dialogue act message,

and

provide a response to the second message, the response being provided based on a response strategy corresponding to the predetermined dialogue act message.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (24)