Visual recognition using user tap locations

1. A computer-implemented method comprising:

receiving (i) a query image and (ii) a user tap location defining an area of interest within the query image;

processing the query image to identify one or more objects in the query image and to crop the query image around each identified object to generate multiple cropped images;

processing a first portion of the query image using a first classifier that identifies one or more first entities within the query image, wherein the first portion of the query image lies outside the area of interest;

processing a second portion of the query image corresponding to one or more of the multiple cropped images that are determined to correspond to the user tap location defining the area of interest using a second classifier that identifies one or more second entities within the one or more of the multiple cropped images, wherein the second classifier has a higher processing power than the first classifier;

obtaining content which provides information about one or more of the identified one or more first entities and the identified one or more second entities which is descriptive of a combination of the identified one or more first entities with the identified one or more second entities such that the obtained content is biased towards the identified one or more second entities; and

providing the obtained content to a user device for display on the user device with the query image.

2. The method of claim 1 , wherein obtaining the content comprises:

identifying a plurality of content items that are associated with the combination of the identified one or more first entities with the identified one or more second entities;

determining a respective relevance score for each of the plurality of content items that are associated with the combination of the identified one or more first entities with the identified one or more second entities; and

selecting one or more of the plurality of content items as the content descriptive of the combination of the identified one or more first entities with the identified one or more second entities based at least in part on their respective relevance scores.

3. The method of claim 2 , wherein the content items are suggested search queries.

4. The method of claim 3 , wherein the suggested search queries are textual search queries.

5. The method of claim 3 , wherein determining a relevance score of a suggested search query comprises determining the relevance score based on: a popularity of the suggested search query, a quality of search results responsive to the suggested search query, or both.

6. The method of claim 1 wherein obtaining the content comprises:

obtaining content based on a classification of the identified one or more second entities that is contextualized using a classification of the identified one or more first entities.

7. The method of claim 3 , further comprising, in response to a user of the user device selecting a particular suggested search query of the plurality of suggested search queries, providing search results responsive to the particular suggested search query to the user device for display on the user device.

8. A system comprising:

one or more computers comprising one or more processors; and

one or more non-transitory storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

receiving (i) a query image and (ii) a user tap location defining an area of interest within the query image;

processing the query image to identify one or more objects in the query image and to crop the query image around each identified object to generate multiple cropped images;

processing a first portion of the query image using a first classifier that identifies one or more first entities within the query image, wherein the first portion of the query image lies outside the area of interest;

processing a second portion of the query image corresponding to one or more of the multiple cropped images that are determined to correspond to the user tap location defining the area of interest using a second classifier that identifies one or more second entities within the one or more of the multiple cropped images, wherein the second classifier has a higher processing power than the first classifier;

obtaining content which provides information about one or more of the identified one or more first entities and the identified one or more second entities which is descriptive of a combination of the identified one or more first entities with the identified one or more second entities such that the obtained content is biased towards the identified one or more second entities; and

providing the obtained content to a user device for display on the user device with the query image.

9. The system of claim 8 , wherein obtaining the content comprises:

identifying a plurality of content items that are associated with the combination of the identified one or more first entities with the identified one or more second entities;

determining a respective relevance score for each of the plurality of content items that are associated with the combination of the identified one or more first entities with the identified one or more second entities; and

selecting one or more of the plurality of content items as the content descriptive of the combination of the identified one or more first entities with the identified one or more second entities based at least in part on their respective relevance scores.

10. The system of claim 9 , wherein the content items are suggested search queries.

11. The system of claim 10 , wherein the suggested search queries are textual search queries.

12. The system of claim 10 , wherein determining a relevance score of a suggested search query comprises determining the relevance score based on: a popularity of the suggested search query, a quality of search results responsive to the suggested search query, or both.

13. The system of claim 8 wherein obtaining the content comprises:

obtaining content based on a classification of the identified one or more second entities that is contextualized using a classification of the identified one or more first entities.

14. The system of claim 10 , further comprising, in response to a user of the user device selecting a particular suggested search query of the plurality of suggested search queries, providing search results responsive to the particular suggested search query to the user device for display on the user device.

15. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving (i) a query image and (ii) a user tap location defining an area of interest within the query image;

processing the query image to identify one or more objects in the query image and to crop the query image around each identified object to generate multiple cropped images;

processing a first portion of the query image using a first classifier that identifies one or more first entities within the query image, wherein the first portion of the query image lies outside the area of interest;

processing a second portion of the query image corresponding to one or more of the multiple cropped images that are determined to correspond to the user tap location defining the area of interest using a second classifier that identifies one or more second entities within the one or more of the multiple cropped images, wherein the second classifier has a higher processing power than the first classifier;

obtaining content which provides information about one or more of the identified one or more first entities and the identified one or more second entities which is descriptive of a combination of the identified one or more first entities with the identified one or more second entities such that the obtained content is biased towards the identified one or more second entities; and

providing the obtained content to a user device for display on the user device with the query image.

16. The non-transitory computer storage media of claim 15 , wherein obtaining the content comprises:

identifying a plurality of content items that are associated with the combination of the identified one or more first entities with the identified one or more second entities;

determining a respective relevance score for each of the plurality of content items that are associated with the combination of the identified one or more first entities with the identified one or more second entities; and

selecting one or more of the plurality of content items as the content descriptive of the combination of the identified one or more first entities with the identified one or more second entities based at least in part on their respective relevance scores.

17. The non-transitory computer storage media of claim 16 , wherein the content items are suggested search queries.

18. The non-transitory computer storage media of claim 17 , wherein the suggested search queries are textual search queries.

19. The non-transitory computer storage media of claim 17 , wherein determining a relevance score of a suggested search query comprises determining the relevance score based on: a popularity of the suggested search query, a quality of search results responsive to the suggested search query, or both.

20. The non-transitory computer storage media of claim 15 wherein obtaining the content comprises:

obtaining content based on a classification of the identified one or more second entities that is contextualized using a classification of the identified one or more first entities.