Smart camera user interface
Implementations of the present disclosure include actions of receiving image data of an image capturing a scene, receiving data describing one or more entities determined from the scene, the one or more entities being determined from the scene, determining one or more actions based on the one or more entities, each action being provided at least partly based on search results from searching the one or more entities, and providing instructions to display an action interface comprising one or more action elements, each action element being to induce execution of a respective action, the action interface being displayed in a viewfinder.
1. A computer-implemented method comprising:
receiving, by one or more processors, image data, the image data being provided from a camera and corresponding to a scene viewed within a viewfinder of the camera, the image data depicting a plurality of entities;
determining, by the one or more processors, one or more annotations for one or more of the plurality of entities depicted by at least a portion of the image data;
respectively determining, by the one or more processors based at least in part on the one or more annotations for each of the one or more entities, one or more purchase actions for the one or more entities; and
displaying, by the one or more processors, within the viewfinder, one or more action cards respectively corresponding to the one or more purchase actions.
2. The computer-implemented method of claim 1 , further comprising:
in response to displaying the one or more action cards, receiving, by the one or more processors, a user input corresponding to the one or more action cards of the one or more purchase actions for the one or more entities; and
executing, by the one or more processors, the one or more purchase actions associated with the one or more action cards.
3. The computer-implemented method of claim 1 , wherein the one or more annotations for at least one of the one or more of the plurality of entities comprises a type for the at least one entity.
4. The computer-implemented method of claim 3 , wherein the type is represented within a structured data graph.
5. The computer-implemented method of claim 4 , wherein respectively determining, by the one or more processors based at least in part on the one or more annotations for each of the one or more entities, one or more purchase actions for the one or more entities comprises:
cross-referencing the entity with objects that have the same type in the structured data graph.
6. The computer-implemented method of claim 4 , wherein the structured data graph comprises data specific to a user.
7. A computing system comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the computing system to perform operations comprising:
receiving image data, the image data being provided from a camera and corresponding to a scene viewed within a viewfinder of the camera, the image data depicting a plurality of entities;
determining one or more annotations for one or more of the plurality of entities depicted by at least a portion of the image data;
respectively determining, based at least in part on the one or more annotations for each of the one or more entities, one or more purchase actions for the one or more entities; and
displaying one or more action cards respectively corresponding to the one or more purchase actions.
8. The computing system of claim 7 , further comprising:
in response to displaying the one or more action cards, receiving a user input corresponding to the one or more action cards of the one or more purchase actions for the one or more entities; and
executing, by the one or more processors, the one or more purchase actions associated with the one or more action cards.
9. The computing system of claim 7 , wherein the one or more annotations for at least one of the one or more of the plurality of entities comprises a type for the at least one entity.
10. The computing system of claim 9 , wherein the type is represented within a structured data graph.
11. The computing system of claim 10 , wherein respectively determining, for each of the one or more entities, one or more purchase actions for the one or more entities comprises:
cross-referencing the entity with objects that have the same type in the structured data graph.
12. The computing system of claim 10 , wherein the structured data graph comprises data specific to a user.
13. The computing system of claim 7 , wherein the image data comprises an image captured by a user.
14. The computing system of claim 7 , wherein the viewfinder is within a mobile application.
15. One or more tangible, non-transitory, computer-readable media that collectively store instructions when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:
receiving image data, the image data being provided from a camera and corresponding to a scene viewed within a viewfinder of the camera, the image data depicting a plurality of entities;
determining one or more annotations for one or more of the plurality of entities depicted by at least a portion of the image data;
respectively determining, based at least in part on the one or more annotations for each of the one or more entities, one or more purchase actions for the one or more entities; and
displaying one or more action cards respectively corresponding to the one or more purchase actions.
16. The one or more tangible, non-transitory, computer-readable media of claim 15 , further comprising:
in response to displaying the one or more action cards, receiving, by the one or more processors, a user input corresponding to the one or more action cards of the one or more purchase actions for the one or more entities; and
executing, by the one or more processors, the one or more purchase actions associated with the one or more action cards.
17. The one or more tangible, non-transitory, computer-readable media of claim 16 , wherein the one or more annotations for at least one of the one or more of the plurality of entities comprises a type for the at least one entity.
18. The one or more tangible, non-transitory, computer-readable media of claim 17 , wherein the type is represented within a structured data graph.
19. The one or more tangible, non-transitory, computer-readable media of claim 18 , wherein respectively determining, based at least in part on the one or more annotations for each of the one or more entities, one or more purchase actions for the one or more entities comprises:
cross-referencing the entity with objects that have the same type in the structured data graph.
20. The one or more tangible, non-transitory, computer-readable media of claim 18 , wherein the structured data graph comprises data specific to a user.