IP Library › Granted Patent US 11,423,886
Granted Patent B2
US 11,423,886 · App. 16/879,643 · Granted Aug 23, 2022

Task flow identification based on user intent

Inventors: Thomas Robert Gruber (Santa Cruz, CA); Adam John Cheyer (Oakland, CA); Dag Kittlaus (San Jose, CA); Didier Rene Guzzoni (Montsur-Rolle, CH); Christopher Dean Brigham (San Jose, CA); Richard Donald Giuli (Arroyo Grande, CA); Marcello Bastea-Forte (San Francisco, CA); Harry Joseph Saddler (Berkeley, CA)
Assignee: Apple Inc.
G10L15/1815B60K35/00G06F3/167G06F9/54G06F16/3329G06F16/3344G06F16/9537G06F40/279G06F40/30G06F40/40G10L13/02G10L13/08G10L15/183G10L15/22G10L15/26G10L21/06H04M1/6091H04M1/7243H04M1/72448H04M1/72484G10L13/00G10L15/1822
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,423,886
App. No.
16/879,643
Filed
May 20, 2020
Granted
Aug 23, 2022
Kind
B2
Art Unit
2658
USPC
704/275
Abstract

The intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Claims (112)

1. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving input from a user;

determining an intent corresponding to the received input;

performing a first task based on the determined intent;

providing a first output associated with the performed task;

in accordance with a detection of a user selection corresponding to the first output:

providing a second output including at least one additional task corresponding to the performed task; and

in accordance with a detection that no user selection corresponding to the first output has occurred:

providing a third output including at least one option to select a task parameter associated with the first task.

2. The device of claim 1 , the one or more programs further including instructions for:

receiving a second input from the user, wherein the second input corresponds to a selection of a task from the at least one provided additional task; and

performing a second task based on the second input.

3. The device of claim 1 , wherein the at least one additional task includes an interaction with a remote source over a network connection.

4. The device of claim 1 , wherein the at least one additional task includes an interaction with information stored on the electronic device.

5. The device of claim 1 , wherein the at least one additional task includes a task to save information corresponding to the first task.

6. The device of claim 1 , wherein the at least one additional task includes a task to send information to another user, wherein the information corresponds to the first task.

7. The device of claim 1 , wherein the at least one additional task includes a task to display map information corresponding to the first task.

8. The device of claim 1 , wherein the at least one additional task includes a task to initiate a phone call to a contact associated with the first task.

9. The device of claim 1 , wherein the at least one additional task includes a task to set an alarm associated with the first task.

10. The device of claim 1 , wherein the at least one additional task includes a task to initiate a reservation procedure associated with the first task.

11. The device of claim 1 , wherein the at least one additional task includes a task to create a note file associated with the first task.

12. The device of claim 1 , wherein the second output includes an audible representation of at least one additional task corresponding to the performed task.

13. The device of claim 1 , the one or more programs further including instructions for:

receiving a second user input including a speech input; and

determining a user intent of the second speech input based on the second output including at least one additional task corresponding to the performed task.

14. The device of claim 1 , wherein at least one of the first task and the at least one additional task includes performing a task on a secondary device.

15. The device of claim 1 , wherein the task parameter is associated with a domain corresponding to the first task.

16. The device of claim 1 , wherein determining an intent corresponding to the received input comprises:

obtaining context information including a history of interactions between the user and the electronic device; and

determining the intent based on the context information.

17. The device of claim 16 , the one or more programs further including instructions for:

prior to receiving the input from the user, receiving a speech input from the user within a predetermined time of the received input; and

processing the received speech input to obtain the context information.

18. The device of claim 16 , the one or more programs further including instructions for:

prior to receiving the input from the user, receiving a non-speech input from the user within a predetermined time of the received input; and

processing the received non-speech input to obtain the context information.

19. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the device to:

receive input from a user;

determine an intent corresponding to the received input;

perform a first task based on the determined intent;

provide a first output associated with the performed task;

in accordance with a detection of a user selection corresponding to the first output:

provide a second output including at least one additional task corresponding to the performed task; and

in accordance with a detection that no user selection corresponding to the first output has occurred:

provide a third output including at least one option to select a task parameter associated with the first task.

20. The computer readable storage medium of claim 19 , wherein the one or more programs cause the device to:

receive a second input from the user, wherein the second input corresponds to a selection of a task from the at least one provided additional task; and

perform a second task based on the second input.

21. The computer readable storage medium of claim 19 , wherein the at least one additional task includes an interaction with a remote source over a network connection.

22. The computer readable storage medium of claim 19 , wherein the at least one additional task includes an interaction with information stored on the electronic device.

23. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to save information corresponding to the first task.

24. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to send information to another user, wherein the information corresponds to the first task.

25. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to display map information corresponding to the first task.

26. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to initiate a phone call to a contact associated with the first task.

27. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to set an alarm associated with the first task.

28. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to initiate a reservation procedure associated with the first task.

29. The computer readable storage medium of claim 19 , wherein the at least one additional task includes a task to create a note file associated with the first task.

30. The computer readable storage medium of claim 19 , wherein the second output includes an audible representation of at least one additional task corresponding to the performed task.

31. The computer readable storage medium of claim 19 , wherein the one or more programs cause the device to:

receive a second user input including a speech input; and

determine a user intent of the second speech input based on the second output including at least one additional task corresponding to the performed task.

32. The computer readable storage medium of claim 19 , wherein at least one of the first task and the at least one additional task includes performing a task on a secondary device.

33. The computer readable storage medium of claim 19 , wherein the task parameter is associated with a domain corresponding to the first task.

34. The computer readable storage medium of claim 19 , wherein determining an intent corresponding to the received input comprises:

obtaining context information including a history of interactions between the user and the electronic device; and

determining the intent based on the context information.

35. The computer readable storage medium of claim 34 , wherein the one or more programs cause the device to:

prior to receiving the input from the user, receiving a speech input from the user within a predetermined time of the received input; and

processing the received speech input to obtain the context information.

36. The computer readable storage medium of claim 34 , wherein the one or more programs cause the device to:

prior to receiving the input from the user, receiving a non-speech input from the user within a predetermined time of the received input; and

processing the received non-speech input to obtain the context information.

37. A computer-implemented method, comprising:

at an electronic device with one or more processors and memory:

receiving input from a user;

determining an intent corresponding to the received input;

performing a first task based on the determined intent;

providing a first output associated with the performed task;

in accordance with a detection of a user selection corresponding to the first output:

providing a second output including at least one additional task corresponding to the performed task; and

in accordance with a detection that no user selection corresponding to the first output has occurred:

providing a third output including at least one option to select a task parameter associated with the first task.

38. The method of claim 37 , comprising:

receiving a second input from the user, wherein the second input corresponds to a selection of a task from the at least one provided additional task; and

performing a second task based on the second input.

39. The method of claim 37 , wherein the at least one additional task includes an interaction with a remote source over a network connection.

40. The method of claim 37 , wherein the at least one additional task includes an interaction with information stored on the electronic device.

41. The method of claim 37 , wherein the at least one additional task includes a task to save information corresponding to the first task.

42. The method of claim 37 , wherein the at least one additional task includes a task to send information to another user, wherein the information corresponds to the first task.

43. The method of claim 37 , wherein the at least one additional task includes a task to display map information corresponding to the first task.

44. The method of claim 37 , wherein the at least one additional task includes a task to initiate a phone call to a contact associated with the first task.

45. The method of claim 37 , wherein the at least one additional task includes a task to set an alarm associated with the first task.

46. The method of claim 37 , wherein the at least one additional task includes a task to initiate a reservation procedure associated with the first task.

47. The method of claim 37 , wherein the at least one additional task includes a task to create a note file associated with the first task.

48. The method of claim 37 , wherein the second output includes an audible representation of at least one additional task corresponding to the performed task.

49. The method of claim 37 , comprising:

receiving a second user input including a speech input; and

determining a user intent of the second speech input based on the second output including at least one additional task corresponding to the performed task.

50. The method of claim 37 , wherein at least one of the first task and the at least one additional task includes performing a task on a secondary device.

51. The method of claim 37 , wherein the task parameter is associated with a domain corresponding to the first task.

52. The method of claim 37 , wherein determining an intent corresponding to the received input comprises:

obtaining context information including a history of interactions between the user and the electronic device; and

determining the intent based on the context information.

53. The method of claim 52 , comprising:

prior to receiving the input from the user, receiving a speech input from the user within a predetermined time of the received input; and

processing the received speech input to obtain the context information.

54. The method of claim 52 , comprising:

prior to receiving the input from the user, receiving a non-speech input from the user within a predetermined time of the received input; and

processing the received non-speech input to obtain the context information.

Continuity (5)
Continuation 15394162 · Dec 29, 2016
Continuation 13492809 · Jun 9, 2012
Continuation 12987982 · Jan 10, 2011
Provisional Application 61295774 · Jan 18, 2010
Related Publication 20200279556A1 · Sep 3, 2020
Cited By (6)
US 12,346,668 US 12,373,391 US 12,400,082 US 12,431,128 US 12,436,970 US 12,681,587