IP Library › Granted Patent US 11,521,114
Granted Patent B2
US 11,521,114 · App. 16/388,015 · Granted Dec 6, 2022

Visualization of training dialogs for a conversational bot

Inventors: Lars H. Liden (Seattle, WA); Swadheen K. Shukla (Seattle, WA); Shahin Shayandeh (Seattle, WA); Matthew D. Mazzola (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
G06N20/00G06F3/0482G06F16/9027G06N5/04H04L51/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,521,114
App. No.
16/388,015
Granted
Dec 6, 2022
Kind
B2
Abstract

This document relates to creating and/or updating a chatbot using a graphical user interface. For example, training dialogs for a chatbot can be displayed in a tree form on a graphical user interface. Based at least on interactions between a developer and the graphical user interface, the training dialogs can be modified in the tree form, and training dialogs can be updated based on the modifications provided on the tree form via the graphical user interface.

Claims (51)

1. A method comprising:

accessing stored training dialogs employed to train at least one of an entity extractor or a response model of a conversational system;

receiving a selection to display the training dialogs in a tree form;

generating the tree form of the training dialogs based at least on the received selection, the tree form representing the stored training dialogs as nodes with branches representing decision points, wherein generating the tree form includes merging two or more of the training dialogs into a common node based at least upon the two or more of the training dialogs sharing a common state and a common action;

displaying the tree form of the training dialogs, including the common node and one or more other nodes of the tree form; and

updating the stored training dialogs based at least on modifications received to the tree form to obtain updated training dialogs, the updated training dialogs providing a basis for further training of the entity extractor or the response model of the conversational system.

2. The method of claim 1 , further comprising:

displaying a listing of the stored training dialogs as a textual listing on a graphical user interface; and

responsive to the received selection to display the training dialogs in a tree form, displaying the generated tree form on the graphical user interface.

3. The method of claim 2 , wherein generating the tree form comprises:

generating a particular node that depicts one or more inputs, one or more current states for the conversational system, and one or more actions associated with the particular node.

4. The method of claim 1 , further comprising:

receiving at least one of the modifications to the tree form via a user selection to change an attribute associated with a particular node of the tree form.

5. The method of claim 1 , wherein the two or more of the training dialogs that are merged into the common node specify different values for the same entity.

6. The method of claim 1 , wherein the generating the tree form of the training dialogs further comprises:

determining that two or more other training dialogs result in a conflict;

highlighting, on the generated tree form, the conflict between the two or more other training dialogs; and

updating the generated tree form responsive to user input resolving the conflict between the two or more other training dialogs.

7. The method of claim 1 , further comprising receiving new training dialogs through the tree form.

8. The method of claim 1 , wherein one or more of the decision points are based at least on machine learning.

9. The method of claim 1 , wherein one or more of the decision points are based at least on declarative rules.

10. The method of claim 1 , wherein the tree form displays branches representing decision points based at least on declarative rules simultaneously with other branches representing decision points based at least on machine learning.

11. The method of claim 1 , further comprising:

retrieving the updated stored training dialogs that reflect the modifications made to the tree form responsive to user interaction with the conversational system; and

outputting, by the conversational system, a natural language response to the user interaction based at least on the updated stored training dialogs.

12. A method comprising:

displaying on a graphical user interface (GUI) a visual tree representing training dialogs employed to train at least one of an entity extractor or a response model of a conversational system, wherein the visual tree includes a common node representing two or more of the training dialogs that have been merged based at least on the two or more of the training dialogs sharing a common state and a common action;

receiving a modification to one or more of the training dialogs via the GUI;

propagating the modification to other training dialogs in the visual tree that are dependent on the modified one or more training dialogs; and

updating a database of training dialogs based at least on the received modification, the updated database of training dialogs providing a basis for further training of the entity extractor or the response model of the conversational system.

13. The method of claim 12 , further comprising:

determining whether the modification to the one or more training dialogs results in a conflict with another training dialog; and

providing an indication via the GUI indicating the conflict resulting from the modification.

14. The method of claim 13 , further comprising updating the visual tree when a user resolves the conflict.

15. The method of claim 12 , further comprising receiving a selection to display the training dialogs in a textual list form, wherein the received modification is reflected in the textual list form.

16. The method of claim 12 , further comprising indicating, via the GUI, whether a decision in the visual tree resulted from declarative rules or machine learning.

17. A system comprising:

a display of the system configured to depict a graphical user interface;

a processor; and

a storage memory storing computer-readable instructions, which when executed by the processor, cause the processor to:

display, via the graphical user interface, data associated with training dialogs that represent interactions with a chatbot, the training dialogs employed to train a machine learning model used by the chatbot;

receive an indication from a user of the system to display the data associated with the training dialogs in a tree form;

determine whether decisions depicted in the tree form are based at least on declarative rules or based at least on machine learning by the machine learning model used by the chatbot; and

display first decisions that are based at least on declarative rules differently from second decisions that are based at least on machine learning by the machine learning model used by the chatbot.

18. The system of claim 17 , wherein the first decisions that are based at least on declarative rules are displayed using a different color than the second decisions that are based at least on machine learning by the machine learning model used by the chatbot.

19. The system of claim 17 , wherein first nodes of the tree form that are based at least on declarative rules are displayed using a different icon than second nodes of the tree form that are based at least on machine learning by the machine learning model used by the chatbot.

20. The system of claim 17 , wherein computer-readable instructions, when executed by the processor, further cause the processor to:

receive a selection on a node of the tree form; and

display editable data associated with the selected node.

21. The method of claim 1 , further comprising:

training at least one of the entity extractor or the response model based at least on the updated training dialogs.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2019
From: LIDEN, LARS H.; SHUKLA, SWADHEEN K.; SHAYANDEH, SHAHIN; MAZZOLA, MATTHEW D.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 049259/0025 →
Continuity (1)
Related Publication 20200334568A1 · Oct 22, 2020