IP Library › Granted Patent US 11,961,509
Granted Patent B2
US 11,961,509 · App. 16/839,308 · Granted Apr 16, 2024

Training a user-system dialog in a task-oriented dialog system

Inventors: Swadheen Kumar Shukla (Seattle, WA); Lars Hasso Liden (Seattle, WA); Thomas Park (Seattle, WA); Matthew David Mazzola (Seattle, WA); Shahin Shayandeh (Seattle, WA); Jianfeng Gao (Woodinville, WA); Eslam Kamal Abdelreheem (Sammamish, WA)
Assignee: Microsoft Technology Licensing, LLC
G10L15/063G06N3/044G06N3/049G06N3/08G10L15/16G10L15/22G10L25/30G10L2015/0635G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,961,509
App. No.
16/839,308
Granted
Apr 16, 2024
Kind
B2
Abstract

Methods and systems are disclosed for improving dialog management for task-oriented dialog systems. The disclosed dialog builder leverages machine teaching processing to improve development of dialog managers. In this way, the dialog builder combines the strengths of both rule-based and machine-learned approaches to allow dialog authors to: (1) import a dialog graph developed using popular dialog composers, (2) convert the dialog graph to text-based training dialogs, (3) continuously improve the trained dialogs based on log dialogs, and (4) generate a corrected dialog for retraining the machine learning.

Claims (58)

1. A computer-implemented method for correcting a dialog, the method comprising:

receiving a first dialog graph comprising a plurality of nodes and at least one edge connecting two nodes of the plurality of nodes, wherein the first dialog graph represents a dialog flow including each of the plurality of nodes defining an action associated with the corresponding node and the at least one edge defining a condition linking the two nodes, wherein a first path connects at least a first preceding node to at least a first subsequent node of the plurality of nodes through one or more edges, and wherein a second path connects at least the first preceding node to at least a second subsequent node of the plurality of nodes;

converting the first path of the first dialog graph into a first text-based dialog and the second path of the first dialog graph into a second text-based dialog, wherein the first text-based dialog and the second text-based dialog are in a data format adapted for training a neural network and represent the dialog flow of the first dialog graph;

training the neural network based at least on the first text-based dialog and the second text-based dialog as training data;

receiving a log dialog, wherein the log dialog is generated based on executing the neural network to deploy a dialog, wherein a conversation thread associated with the deployed dialog is included in the log dialog;

identifying an exception in the log dialog;

converting at least a portion of the log dialog into a second dialog graph;

receiving an edit directly to the conversation thread of the deployed dialog in the log dialog via an interactive dialog editor tool to mitigate the exception and to create a corrected dialog;

retraining the neural network based at least on the corrected dialog; and

updating the second dialog graph associated with the log dialog based on the corrected dialog.

2. The computer-implemented method of claim 1 , wherein the neural network is retrained based at least on the first and second text-based dialogs and the corrected dialog.

3. The computer-implemented method of claim 1 , wherein the log dialog comprises at least one user input and at least one system response.

4. The computer-implemented method of claim 3 , wherein the identified exception is associated with the at least one system response.

5. The computer-implemented method of claim 1 , the method further comprising:

receiving an edit to the second dialog graph to mitigate the exception and to create a corrected dialog graph;

converting each path of the corrected dialog graph into a corrected text-based dialog to generate a plurality of corrected text-based dialogs; and

training the neural network based on the plurality of corrected of text-based dialogs.

6. The computer-implemented method of claim 5 , wherein the edit to the second dialog graph comprises creating at least one of a new edge or a new node in the second dialog graph.

7. The computer-implemented method of claim 1 , wherein the at least one edge represents a condition for traversing the first path from the first preceding node to the first subsequent node.

8. The computer-implemented method of claim 7 , wherein the first preceding node is associated with a system action, and wherein the system action comprises one of: asking a question, providing a message, calling an application programing interface (API), or constructing a sentence based on a template card with entity values.

9. The computer-implemented method of claim 1 , further comprising:

providing a user interface, wherein the user interface comprises a tool for receiving the edit to the log dialog.

10. A system comprising:

at least one processor; and

at least one memory storing computer-executable instructions that when executed by the at least one processor cause the system to:

receive a first dialog graph comprising a plurality of nodes and at least one edge connecting two nodes of the plurality of nodes, wherein the first dialog graph represents a dialog flow including each of the plurality of nodes defining an action associated with the corresponding node and the at least one edge defining a condition linking the two nodes, wherein a first path connects at least a first preceding node to at least a first subsequent node of the plurality of nodes through one or more edges, and wherein a second path connects at least the first preceding node to at least a second subsequent node of the plurality of nodes;

translate the first path of the first dialog graph into a first text-based dialog and the second path of the first dialog graph into a second text-based dialog, wherein the first text-based dialog and the second text-based dialog are in a data format adapted for training a neural network and represent the dialog flow of the first dialog graph;

train the neural network based at least on the first text-based dialog and the second text-based dialog as training data;

deploy the neural network to generate a log dialog of deployed user-system interactions, wherein at least one deployed dialog is included in the log dialog;

identify an exception associated with the deployed dialog in the log dialog;

convert at least a portion of the log dialog into a second dialog graph;

receive an edit directly to the log dialog via an interactive dialog editor tool to mitigate the exception and to create a corrected dialog;

retrain the neural network based on the plurality of corrected dialog;

update the second dialog graph associated with the log dialog based on the corrected dialog.

11. The system of claim 10 , wherein the neural network is a recurrent neural network.

12. The system of claim 10 , wherein the log dialog comprises at least one user input and at least one system response.

13. The system of claim 10 , wherein the at least one edge represents a condition for traversing the first path from the first preceding node to the first subsequent node.

14. The system of claim 10 , further comprising instructions stored thereon that, when executed by the at least one processor, causes the system to:

receive a second edit to the second dialog graph to mitigate the exception and to create a corrected dialog graph;

convert each path of the corrected dialog graph into a corrected text-based dialog to generate a plurality of corrected text-based dialogs; and

train the neural network based on the plurality of corrected of text-based dialogs,

wherein the second edit to the second dialog graph comprises creating at least one of a new edge or a new node in the second dialog graph.

15. A computer storage medium storing computer-executable instructions that when executed a processor cause a computer system to:

receive a first dialog graph comprising a plurality of nodes and at least one edge connecting two nodes of the plurality of nodes, wherein the first dialog graph represents a dialog flow including each of the plurality of nodes defining an action associated with the corresponding node and the at least one edge defining a condition linking the two nodes, wherein a first path connects at least a first preceding node to at least a first subsequent node of the plurality of nodes through one or more edges, and wherein a second path connects at least the first preceding node to at least a second subsequent node of the plurality of nodes;

convert the first path of the first dialog graph into a first text-based dialog and the second path of the first dialog graph into a second text-based dialog, wherein the first text-based dialog and the second text-based dialog are in a data format adapted for training a machine learning system and represent the dialog flow of the first dialog graph;

train the machine learning system based at least on the first text-based dialog and the second text-based dialog as training data;

receive a log dialog, wherein the log dialog is generated based on executing the machine learning system to deploy a dialog, wherein a conversation thread associated with the deployed dialog is included in the log dialog;

cause display of the log dialog in a machine-teaching tool of a user interface;

convert at least a portion of the log dialog into a second dialog graph;

provide an indication of an exception in the log dialog, wherein the exception is associated with the deployed dialog;

receive an edit directly to the conversation thread of the deployed dialog in the log dialog via an interactive dialog editor tool to mitigate the exception;

retrain the machine learning system based on the edited log dialog;

update the second dialog graph associated with the log dialog based on a corrected dialog.

16. The computer storage medium of claim 15 , the computer-executable instructions when executed further causing the computer system to:

cause display of one or more recommendations for mitigating the exception based on the machine-teaching tool; and

receive a selection of at least one recommendation to edit the log dialog to mitigate the identified exception.

17. The computer storage medium of claim 15 , wherein the machine-teaching tool is a neural network.

18. The computer storage medium of claim 15 , wherein the retrained machine learning system is used to update the second dialog graph.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2020
From: SHUKLA, SWADHEEN KUMAR; LIDEN, LARS HASSO; PARK, THOMAS; MAZZOLA, MATTHEW DAVID; SHAYANDEH, SHAHIN; GAO, JIANFENG; ABDELREHEEM, ESLAM KAMAL
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 052304/0492 →
Continuity (1)
Related Publication 20210312904A1 · Oct 7, 2021
Cited By (1)
US 12,412,571