IP Library Granted Patent US 11,360,739
Granted Patent B2
US 11,360,739 · App. 16/592,906 · Granted Jun 14, 2022

User activity shortcut suggestions

Inventors: John L. Blatz (San Francisco, CA); Andrew William Malta (San Francisco, CA); Jay Moon (Palo Alto, CA); Pallavika Ramaswamy (Cupertino, CA); Ari Weinstein (San Francisco, CA)
Assignee: Apple Inc.
G06F3/167G06F9/451G10L15/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,360,739
App. No.
16/592,906
Granted
Jun 14, 2022
Kind
B2
Abstract

Systems and processes for accelerating task performance are provided. An example method includes, at an electronic device with a display, displaying a candidate shortcut affordance associated with a user activity, detecting a first set of inputs corresponding to a selection of the candidate shortcut affordance, in response to detecting the first set of inputs, displaying a first set of candidate task affordances, detecting a second set of inputs corresponding to a selection of a candidate task affordance associated with a first task, displaying a second set of candidate task affordances, detecting a third set of inputs corresponding to a selection of a candidate task affordance associated with a second task, and in response to detecting the second set of inputs and the third set of inputs, associating the first task and the second task with a task sequence for a voice shortcut corresponding to the user activity.

Claims (109)

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:

present a plurality of candidate user activities for a voice shortcut corresponding to a user activity, wherein the plurality of candidate user activities includes a first user activity previously performed by a user of the first electronic device;

receive a first user input selecting the first user activity from the plurality of candidate user activities;

determine, based on at least one previous performance of the first user activity by the user of the first electronic device, contextual data present when the user previously performed the first user activity;

in response to receiving the first user input selecting the first user activity, generate, based on the first user activity and the contextual data, a first plurality of candidate tasks of a first type and a second plurality of candidate tasks of a second task type different than the first task type, wherein at least one task of the first plurality of candidate tasks was previously performed in a context that matches the contextual data;

provide an output indicating the first plurality of candidate tasks and the second plurality of candidate tasks;

receive a set of user inputs identifying a first candidate task of the first plurality of candidate tasks and a second candidate task of the second plurality of candidate tasks; and

group the first candidate task and the second candidate task into a task sequence for a voice shortcut corresponding to the first user activity.

2. The non-transitory computer-readable storage medium of claim 1 , wherein determining the contextual data present when the user previously performed the first user activity includes:

receiving a second user input identifying the contextual data present when the user previously performed the first user activity.

3. The non-transitory computer-readable storage medium of claim 1 wherein generating, based on the first user activity and the contextual data, the first plurality of candidate tasks and the second plurality of candidate tasks includes:

determining one or more tasks performed in a context that matches the contextual data present when the user previously performed the first user activity; and

providing the first plurality of candidate tasks including the one or more tasks previously performed in a context that matches the contextual data present when the user previously performed the first user activity.

4. The non-transitory computer-readable storage medium of claim 1 , wherein generating, based on the first user activity and the contextual data, the first plurality of candidate tasks and the second plurality of candidate tasks includes:

determining one or more tasks based on a received indication of tasks that are most commonly requested by a plurality of users while engaged in the first user activity; and

providing the first plurality of candidate tasks including the one or more tasks determined based on the indication of tasks that are most commonly requested by a plurality of users while engaged in the first user activity.

5. The non-transitory computer-readable storage medium of claim 1 , wherein one or more tasks of the first plurality of candidate tasks are system-level tasks.

6. The non-transitory computer-readable storage medium of claim 1 , wherein one or more tasks of the first plurality of candidate tasks are application-level tasks.

7. The non-transitory computer-readable storage medium of claim 1 , wherein the first candidate task of the first plurality of candidate tasks is associated with a first task and a first parameter for the first task.

8. The non-transitory computer-readable storage medium of claim 7 , wherein a third candidate task of the first plurality of candidate tasks is associated with the first task and a second parameter for the first task.

9. The non-transitory computer-readable storage medium of claim 1 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises automatically grouping a third task into the task sequence.

10. The non-transitory computer-readable storage medium of claim 1 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises associating context criteria corresponding to the first user activity with the task sequence.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the context criteria corresponding to the first user activity is determined based on the contextual data present when the user previously performed the first user activity.

12. The non-transitory computer-readable storage medium of claim 10 , wherein the context criteria corresponding to the first user activity is determined based on a user input identifying the contextual data present when the user previously performed the first user activity.

13. The non-transitory computer-readable storage medium of claim 10 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises determining if a suggestion notification mode is enabled.

14. The non-transitory computer-readable storage medium of claim 10 , wherein the one or more programs further comprise instructions, which when executed by one or more processors of the first electronic device, cause the first electronic device to:

receive contextual data associated with the electronic device;

determine, based on the received contextual data, whether the context criteria is satisfied; and

in accordance with a determination that the context criteria is satisfied, provide a suggestion notification corresponding to the task sequence, which when selected causes the electronic device to perform the task sequence including the first candidate task and the second candidate task.

15. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by one or more processors of the first electronic device, cause the first electronic device to:

provide a confirmation that the first candidate task and the second candidate task are associated with the task sequence.

16. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further comprise instructions, which when executed by one or more processors of the first electronic device, cause the first electronic device to:

receive a natural-language speech input;

determine whether the natural-language speech input satisfies voice shortcut criteria; and

in accordance with a determination that the natural-language speech input satisfies the voice shortcut criteria, cause the electronic device to perform the task sequence including first candidate task and the second candidate task.

17. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

present a plurality of candidate user activities for a voice shortcut corresponding to a user activity, wherein the plurality of candidate user activities includes a first user activity previously performed by a user of the first electronic device;

receiving a first user input selecting the first user activity from the plurality of candidate user activities;

determining, based on at least one previous performance of the first user activity by the user of the first electronic device, contextual data present when the user previously performed the first user activity;

in response to receiving the first user input selecting the first user activity, generating, based on the first user activity and the contextual data, a first plurality of candidate tasks of a first task type and a second plurality of candidate tasks of a second task type different than the first task type, wherein at least one task of the first plurality of candidate tasks was previously performed in a context that matches the contextual data;

providing an output indicating the first plurality of candidate tasks and the second plurality of candidate tasks;

receiving a set of user inputs identifying a first candidate task of the first plurality of candidate tasks and a second candidate task of the second plurality of candidate tasks; and

grouping the first candidate task and the second candidate task into a task sequence for a voice shortcut corresponding to the first user activity.

18. The electronic device of claim 17 , wherein determining the contextual data present when the user previously performed the first user activity includes:

receiving a second user input identifying the contextual data present when the user previously performed the first user activity.

19. The electronic device of claim 17 , wherein generating, based on the first user activity and the contextual data, the first plurality of candidate tasks and the second plurality of candidate tasks includes:

determining one or more tasks performed in a context that matches the contextual data present when the user previously performed the first user activity; and

providing the first plurality of candidate tasks including the one or more tasks performed in a context that matches the contextual data present when the user previously performed the first user activity.

20. The electronic device of claim 17 , wherein generating, based on the first user activity and the contextual data, the first plurality of candidate tasks and the second plurality of candidate tasks includes:

determining one or more tasks based on a received indication of tasks that are most commonly requested by a plurality of users while engaged in the first user activity; and

providing the first plurality of candidate tasks including the one or more tasks determined based on the indication of tasks that are most commonly requested by a plurality of users while engaged in the first user activity.

21. The electronic device of claim 17 , wherein one or more tasks of the first plurality of candidate tasks are system-level tasks.

22. The electronic device of claim 17 , wherein one or more tasks of the first plurality of candidate tasks are application-level tasks.

23. The electronic device of claim 17 , wherein the first candidate task of the first plurality of candidate tasks is associated with a first task and a first parameter for the first task.

24. The electronic device of claim 23 , wherein a third candidate task of the first plurality of candidate tasks is associated with the first task and a second parameter for the first task.

25. The electronic device of claim 17 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises automatically grouping a third task into the task sequence.

26. The electronic device of claim 17 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises associating context criteria corresponding to the user activity with the task sequence.

27. The electronic device of claim 26 , wherein the context criteria corresponding to the first user activity is determined based on the contextual data associated with the first user activity.

28. The electronic device of claim 26 , wherein the context criteria corresponding to the first user activity is determined based on a user input identifying the contextual data present when the user previously performed the first user activity.

29. The electronic device of claim 26 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises determining if a suggestion notification mode is enabled.

30. The electronic device of claim 26 , wherein the one or more programs further include instructions for:

receiving contextual data associated with the electronic device;

determining, based on the received contextual data, whether the context criteria is satisfied; and

in accordance with a determination that the context criteria is satisfied, providing a suggestion notification corresponding to the task sequence, which when selected causes the electronic device to perform the task sequence including the first candidate task and the second candidate task.

31. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

providing a confirmation that the first candidate task and the second candidate task are associated with the task sequence.

32. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

receiving a natural-language speech input;

determining whether the natural-language speech input satisfies voice shortcut criteria; and

in accordance with a determination that the natural-language speech input satisfies the voice shortcut criteria, causing the electronic device to perform the task sequence including the first candidate task and the second candidate task.

33. A method, comprising:

at an electronic device with a display and a touch-sensitive surface:

present a plurality of candidate user activities for a voice shortcut corresponding to a user activity, wherein the plurality of candidate user activities includes a first user activity previously performed by a user of the first electronic device;

receiving a first user input selecting the first user activity from the plurality of candidate user activities;

determining, based on at least one previous performance of the first user activity by the user of the first electronic device, contextual data present when the user previously performed the first user activity;

in response to receiving the first user input selecting the first user activity, generating, based on the first user activity and the contextual data, a first plurality of candidate tasks of a first task type and a second plurality of candidate tasks of a second task type different than first task type, wherein at least one task of the first plurality of candidate tasks was previously performed in a context that matches the contextual data;

providing an output indicating the first plurality of candidate tasks and the second plurality of candidate tasks;

receiving a set of user inputs identifying a first candidate task of the first plurality of candidate tasks and a second candidate task of the second plurality of candidate tasks; and

grouping the first candidate task and the second candidate task into a task sequence for a voice shortcut corresponding to the first user activity.

34. The method of claim 33 , wherein determining the contextual data associated with the first user activity includes:

receiving a second user input identifying the contextual data present when the user previously performed the first user activity.

35. The method of claim 33 , wherein generating, based on the first user activity and the contextual data, the first plurality of candidate tasks and the second plurality of candidate tasks includes:

determining one or more tasks performed in a context that matches the contextual data present when the user previously performed the first user activity; and

providing the first plurality of candidate tasks including the one or more tasks performed in a context that matches the contextual data present when the user previously performed the first user activity.

36. The method of claim 33 , wherein generating, based on the first user activity and the contextual data, the first plurality of candidate tasks and the second plurality of candidate tasks includes:

determining one or more tasks based on a received indication of tasks that are most commonly requested by a plurality of users while engaged in the first user activity; and

providing the first plurality of candidate tasks including the one or more tasks determined based on the indication of tasks that are most commonly requested by a plurality of users while engaged in the first user activity.

37. The method of claim 33 , wherein one or more tasks of the first plurality of candidate tasks are system-level tasks.

38. The method of claim 33 , wherein one or more tasks of the first plurality of candidate tasks are application-level tasks.

39. The method of claim 33 , wherein the first candidate task of the first plurality of candidate tasks is associated with a first task and a first parameter for the first task.

40. The method of claim 39 , wherein a third candidate task of the first plurality of candidate tasks is associated with the first task and a second parameter for the first task.

41. The method of claim 33 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises automatically grouping a third task into the task sequence.

42. The method of claim 33 , wherein grouping the first candidate task and the second candidate task into the task sequence for the voice shortcut corresponding to the first user activity further comprises associating context criteria corresponding to the first user activity with the task sequence.

43. The method of claim 42 , wherein the context criteria corresponding to the first user activity is determined based on the contextual data present when the user previously performed the first user activity.

44. The method of claim 42 , wherein the context criteria corresponding to the first user activity is determined based on a user input identifying the contextual data present when the user previously performed the first user activity.

45. The method of claim 42 , wherein associating the first candidate task and the second candidate task with the task sequence for the voice shortcut corresponding to the first user activity further comprises determining if a suggestion notification mode is enabled.

46. The method of claim 42 , further comprising:

receiving contextual data associated with the electronic device;

determining, based on the received contextual data, whether the context criteria is satisfied; and

in accordance with a determination that the context criteria is satisfied, providing a suggestion notification corresponding to the task sequence, which when selected causes the electronic device to perform the task sequence including the first candidate task and the second candidate task.

47. The method of claim 33 , further comprising:

providing a confirmation that the first candidate task and the second candidate task are associated with the task sequence.

48. The method of claim 33 , further comprising:

receiving a natural-language speech input;

determining whether the natural-language speech input satisfies voice shortcut criteria; and

in accordance with a determination that the natural-language speech input satisfies the voice shortcut criteria, causing the electronic device to perform the task sequence including the first candidate task and the second candidate task.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2020
From: BLATZ, JOHN L.; MALTA, ANDREW WILLIAM; MOON, JAY; RAMASWAMY, PALLAVIKA; WEINSTEIN, ARI
To: APPLE INC.
Reel/Frame 052593/0213 →
Continuity (2)
Provisional Application 62855631 · May 31, 2019
Related Publication 20200379726A1 · Dec 3, 2020
Cited By (1)
US 12,518,752