Contextual action predictions
This relates generally to intelligent automated assistants and, more specifically, to provide intelligent contextual action predictions by the intelligent automated assistants. An example method includes, at an electronic device displaying a content to a user, receiving a user input on a display area of the displayed content, determining a content type associated with a content object of the displayed content in response to receiving the user input, extracting one or more data items from the determined content type, determining one or more action types of each of the one or more data items and one or more action parameters for the each of the action types, determining one or more suggestive actions based on the determined one or more actions types; and presenting, to the user, at least one suggestive action from the one or more suggestive actions.
1. A method performed on an electronic device, the method comprising:
while displaying content on the electronic device:
receiving a user input within a display area of the displayed content; and
in response to receiving the user input:
determining a content type associated with a content object of the displayed content associated with the user input, wherein the content type is determined based on a content graph of the content object;
extracting, from the content graph of the content object, one or more data items associated with the content object based on the determined content type;
after extracting the one or more data items, determining, from the content graph of the content object, one or more action types of each of the one or more data items and one or more action parameters for each of the one or more action types, wherein the content graph includes one or more respective representations of the one or more action types;
determining one or more suggestive actions based on the determined one or more action types; and
presenting, to a user, at least one suggestive action from the one or more suggestive actions.
2. The method of claim 1 , wherein the content graph represents one or more semantic properties of the content object, wherein the one or more semantic properties are used to determine the one or more suggestive actions for the content object.
3. The method of claim 1 , wherein the user input corresponds to a gesture performed on the displayed content.
4. The method of claim 1 , wherein extracting the one or more data items is based on a source application associated with the content object.
5. The method of claim 1 , wherein each of the one or more suggestive actions is an output-producing action or a non-output producing action.
6. The method of claim 1 , wherein determining the one or more suggestive actions is based on a source application associated with the content object and time of the user input.
7. The method of claim 1 , wherein each of the one or more data items is assigned a weight based on one or more historical interactions involving the each of the one or more data items.
8. The method of claim 1 , wherein the one or more action parameters for each of the one or more action types are determined based on a user-specific information or a non-user specific information available on the electronic device.
9. The method of claim 1 , wherein the content object is a text object, and wherein the one or more suggestive actions for the text object is at least one of markup the text object, share the text object, electronically mail the text object, or translate the text object.
10. The method of claim 1 , wherein presenting the at least one suggestive actions comprises:
determining a rank of each of the at least one suggestive actions; and
presenting the at least one suggestive actions in a sequence based on the rank associated with each of the at least one suggestive actions.
11. The method of claim 1 , wherein the one or more suggestive actions include at least one of a single action, a sequence of plurality of actions, or a composite action.
12. The method of claim 1 , wherein the one or more suggestive actions include a shortcut to one or more predefined actions.
13. The method of claim 1 , wherein the determined content type of the content object is at least one of text or an image within a user interface displayed on the electronic device.
14. An electronic device comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving a user input within a display area of a displayed content on the electronic device; and
in response to receiving the user input:
determining a content type associated with a content object of the displayed content associated with the user input, wherein the content type is determined based on a content graph of the content object;
extracting, from the content graph of the content object, one or more data items associated with the content object based on the determined content type;
after extracting the one or more data items, determining, from the content graph of the content object, one or more action types of each of the one or more data items and one or more action parameters for each of the one or more action types, wherein the content graph includes one or more respective representations of the one or more action types;
determining one or more suggestive actions based on the determined one or more action types; and
presenting, to a user, at least one suggestive action from the one or more suggestive actions.
15. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:
receiving a user input within a display area of a displayed content on the electronic device; and
in response to receiving the user input:
determining a content type associated with a content object of the displayed content associated with the user input, wherein the content type is determined based on a content graph of the content object;
extracting, from the content graph of the content object, one or more data items associated with the content object based on the determined content type;
after extracting the one or more data items, determining, from the content graph of the content object, one or more action types of each of the one or more data items and one or more action parameters for each of the one or more action types, wherein the content graph includes one or more respective representations of the one or more action types;
determining one or more suggestive actions based on the determined one or more action types; and
presenting, to a user, at least one suggestive action from the one or more suggestive actions.
16. The electronic device of claim 14 , wherein the content graph represents one or more semantic properties of the content object, wherein the one or more semantic properties are used to determine the one or more suggestive actions for the content object.
17. The electronic device of claim 14 , wherein the user input corresponds to a gesture performed on the displayed content.
18. The electronic device of claim 14 , wherein extracting the one or more data items is based on a source application associated with the content object.
19. The electronic device of claim 14 , wherein each of the one or more suggestive actions is an output-producing action or a non-output producing action.
20. The electronic device of claim 14 , wherein determining the one or more suggestive actions is based on a source application associated with the content object and time of the user input.
21. The electronic device of claim 14 , wherein each of the one or more data items is assigned a weight based on one or more historical interactions involving the each of the one or more data items.
22. The electronic device of claim 14 , wherein the one or more action parameters for each of the one or more action types are determined based on a user-specific information or a non-user specific information available on the electronic device.
23. The electronic device of claim 14 , wherein the content object is a text object, and wherein the one or more suggestive actions for the text object is at least one of markup the text object, share the text object, electronically mail the text object, or translate the text object.
24. The electronic device of claim 14 , wherein presenting the at least one suggestive actions comprises:
determining a rank of each of the at least one suggestive actions; and
presenting the at least one suggestive actions in a sequence based on the rank associated with each of the at least one suggestive actions.
25. The electronic device of claim 14 , wherein the one or more suggestive actions include at least one of a single action, a sequence of plurality of actions, or a composite action.
26. The electronic device of claim 14 , wherein the one or more suggestive actions include a shortcut to one or more predefined actions.
27. The electronic device of claim 14 , wherein the determined content type of the content object is at least one of text or an image within a user interface displayed on the electronic device.
28. The non-transitory computer-readable storage medium of claim 15 , wherein the content graph represents one or more semantic properties of the content object, wherein the one or more semantic properties are used to determine the one or more suggestive actions for the content object.
29. The non-transitory computer-readable storage medium of claim 15 , wherein the user input corresponds to a gesture performed on the displayed content.
30. The non-transitory computer-readable storage medium of claim 15 , wherein extracting the one or more data items is based on a source application associated with the content object.
31. The non-transitory computer-readable storage medium of claim 15 , wherein each of the one or more suggestive actions is an output-producing action or a non-output producing action.
32. The non-transitory computer-readable storage medium of claim 15 , wherein determining the one or more suggestive actions is based on a source application associated with the content object and time of the user input.
33. The non-transitory computer-readable storage medium of claim 15 , wherein each of the one or more data items is assigned a weight based on one or more historical interactions involving the each of the one or more data items.
34. The non-transitory computer-readable storage medium of claim 15 , wherein the one or more action parameters for each of the one or more action types are determined based on a user-specific information or a non-user specific information available on the electronic device.
35. The non-transitory computer-readable storage medium of claim 15 , wherein the content object is a text object, and wherein the one or more suggestive actions for the text object is at least one of markup the text object, share the text object, electronically mail the text object, or translate the text object.
36. The non-transitory computer-readable storage medium of claim 15 , wherein presenting the at least one suggestive actions comprises:
determining a rank of each of the at least one suggestive actions; and
presenting the at least one suggestive actions in a sequence based on the rank associated with each of the at least one suggestive actions.
37. The non-transitory computer-readable storage medium of claim 15 , wherein the one or more suggestive actions include at least one of a single action, a sequence of plurality of actions, or a composite action.
38. The non-transitory computer-readable storage medium of claim 15 , wherein the one or more suggestive actions include a shortcut to one or more predefined actions.
39. The non-transitory computer-readable storage medium of claim 15 , wherein the determined content type of the content object is at least one of text or an image within a user interface displayed on the electronic device.