IP Library Granted Patent US 11,948,563
Granted Patent B1
US 11,948,563 · App. 16/917,664 · Granted Apr 2, 2024

Conversation summarization during user-control task execution for assistant systems

Inventors: Xiaohu Liu (Bellevue, WA); Paul Anthony Crook (Newcastle, WA); Zhiguang Wang (Bellevue, WA); Shivani Poddar (Mountain View, CA); Seungwhan Moon (Seattle, WA); Krishna Mittal (Mountain View, CA); Shubham Khandelwal (Sunnyvale, CA); Xin Ming Fan (Menlo Park, CA); Eun Joon Cho (Sunnyvale, CA)
Assignee: Meta Platforms, Inc.
G10L15/22G06F40/56G10L15/063G10L15/08G10L2015/0631G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,948,563
App. No.
16/917,664
Granted
Apr 2, 2024
Kind
B1
Abstract

In one embodiment, a method includes receiving a user request from a client system associated with a user, determining that the user request corresponds to a first suspended task, retrieving a first dialog state of the first suspended task from a dialog history associated with the user, generating a summary of the first suspended task based on the first dialog state using a natural-language generating (NLG) module, and sending instructions to the client system for providing the summary of the first suspended task to the user.

Claims (64)

1. A method comprising, by one or more computing devices:

receiving, from a client system associated with a user, a user request;

determining that the user request corresponds to a first suspended task, wherein the first suspended task was previously paused by the user by a suspend command;

extracting one or more keywords from the user request;

comparing the extracted words to respective dialog states of a plurality of suspended tasks that were previously paused by the user by the suspend command;

selecting the first suspended task corresponding to the user request from among the plurality of suspended tasks, wherein the first suspended task is selected from among the plurality of suspended tasks based on the comparison of the extracted keywords to respective dialog states of a plurality of suspended tasks;

retrieving a first dialog state of the first suspended task from a dialog history associated with the user;

generating, using a natural-language generating (NLG) module, a summary of the first suspended task based on the first dialog state; and

sending, to the client system responsive to the user request, instructions for providing the summary of the first suspended task to the user.

2. The method of claim 1 , further comprising:

calculating a plurality of confidence levels of task continuity associated with the plurality of suspended tasks based on an analysis of respective dialog states of the suspended tasks,

wherein the first suspended task is selected from among the plurality of suspended tasks based on a first confidence level of task continuity associated with the first suspended task.

3. The method of claim 2 , further comprising:

if the first confidence level is below a threshold, then sending, to the client system, a request for user verification of the first suspended task.

4. The method of claim 1 , further comprising:

selecting, from the first dialog state, one or more elements of the first suspended task.

5. The method of claim 4 , wherein the one or more selected elements comprise one or more of:

a user intent associated with the first suspended task,

information associated with the first suspended task provided by the user,

one or more completed portions of the first suspended task, or

one or more incomplete portions of the first suspended task.

6. The method of claim 5 , wherein the summary of the first suspended task comprises references to one or more of the selected one or more elements.

7. The method of claim 1 , further comprising:

sending, to the client system, instructions for providing a prompt for the user to verify the summary.

8. The method of claim 1 , further comprising:

receiving, from the client system and in response to the provided summary, an indication to dismiss the first suspended task.

9. The method of claim 1 , further comprising:

executing the first suspended task, wherein the summary is generated further based on the execution of the first suspended task.

10. The method of claim 1 , further comprising:

determining that the first suspended task corresponds to a most recent suspended task from among a plurality of suspended tasks based on the user request.

11. The method of claim 1 , further comprising:

determining, based on a task policy associated with the first suspended task, one or more incomplete elements from the dialog state of the first suspended task,

wherein the summary further comprises a prompt for the user to provide information associated with one or more of the incomplete elements.

12. The method of claim 1 , further comprising:

selecting a first template corresponding to an intent of the first suspended task based on one or more completed portions of the first suspended task; and

generating the summary based on the first template.

13. The method of claim 12 , wherein generating the summary based on the first template comprises populating the first template with information associated with one or more of the completed portions of the first suspended task stored in the first dialog state.

14. The method of claim 1 , further comprising:

generating, using a machine-learning model, the summary based on one or more completed portions of the first suspended task stored in the first dialog state.

15. The method of claim 14 , further comprising:

generating, using the machine-learning model, one or more templates corresponding to an intent of the first suspended task.

16. The method of claim 1 , wherein the user request comprises audio data from a voice input of the user, and wherein the method further comprises:

generating, by an automatic speech recognition (ASR) module, text corresponding to the voice input;

retrieving elements of the first suspended task from the first dialog state as structured input; and

inputting the text and the structured input into an encoder of the NLG module,

wherein the summary is generated by a decoder based on the input text and the structured input.

17. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, from a client system associated with a user, a user request;

determine that the user request corresponds to a first suspended task, wherein the first suspended task was previously paused by the user by a suspend command;

extract one or more keywords from the user request;

compare the extracted keywords to respective dialog states of a plurality of suspended tasks that were previously paused by the user by the suspend command;

select the first suspended task corresponding to the user request from among the plurality of suspended tasks, wherein the first suspended task is selected from among the plurality of suspended tasks based on the comparison of the extracted keywords to respective dialog states of a plurality of suspended tasks;

retrieve a first dialog state of the first suspended task from a dialog history associated with the user;

generate, using a natural-language generating (NLG) module, a summary of the first suspended task based on the first dialog state; and

send, to the client system responsive to the user request, instructions for providing the summary of the first suspended task to the user.

18. A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

receive, from a client system associated with a user, a user request;

determine that the user request corresponds to a first suspended task, wherein the first suspended task was previously paused by the user by a suspend command;

extract one or more keywords from the user request;

compare the extracted keywords to respective dialog states of a plurality of suspended tasks that were previously paused by the user by the suspend command;

select the first suspended task corresponding to the user request from among the plurality of suspended, wherein the first suspended task is selected from among the plurality of suspended tasks based on the comparison of the extracted keywords to respective dialog states of a plurality of suspended tasks;

retrieve a first dialog state of the first suspended task from a dialog history associated with the user;

generate, using a natural-language generating (NLG) module, a summary of the first suspended task based on the first dialog state; and

send, to the client system responsive to the user request, instructions for providing the summary of the first suspended task to the user.

Assignments (2)
CHANGE OF NAME Recorded Dec 22, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058585/0262 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2020
From: LIU, XIAOHU; CROOK, PAUL ANTHONY; WANG, ZHIGUANG; PODDAR, SHIVANI; MOON, SEUNGWHAN; MITTAL, KRISHNA; KHANDELWAL, SHUBHAM; FAN, XIN MING; CHO, EUN JOON
To: FACEBOOK, INC.
Reel/Frame 053456/0777 →