IP Library Granted Patent US 12,143,676
Granted Patent B2
US 12,143,676 · App. 18/123,565 · Granted Nov 12, 2024

Systems and methods of image searching

Inventor: Jason Witenstein-Weaver (San Jose, CA)
Assignee: GOOGLE LLC
H04N21/4722G06F16/532G06F16/538G06F16/9558G06V10/225G06V10/235G06V20/63G06V30/148H04N21/23418H04N21/4728H04N21/4828H04N21/6582H04N21/812H04N21/8586G06V30/153
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,143,676
App. No.
18/123,565
Granted
Nov 12, 2024
Kind
B2
Abstract

Systems and methods of image searching include receiving content, receiving a request to select an image from content, selecting a plurality of items in the image, retrieving information about the selected item, and providing display data based on the retrieved information.

Claims (62)

1. A system comprising:

a memory; and

a hardware processor that, when executing computer-executable instructions stored in the memory, is configured to:

receive an image from a content item;

determine that the image includes a plurality of items displayed within the image;

transmit a plurality of search queries to one or more sources that search for known items that each correspond to one of the plurality of items displayed in the image while concurrently providing an initial search indication, for each of the plurality of items displayed in the image, indicating a search status for that item;

determine, based on the search, that the plurality of items displayed in the image includes a first item having a known identity and a second item having an unknown identity; and

cause the image from the content item to be presented while providing an auditory feedback of the first item having the known identity and the second item having a plurality of potential identities and while providing an identification indication for each of the plurality of items displayed in the image, wherein the identification indication for the unknown second item includes a query to receive additional information that is used to identify the unknown second item.

2. The system of claim 1 , wherein the initial search indication is modified from an initial visual state indicating that the search for the corresponding known item is ongoing to a modified visual state indicating that the search for the corresponding known item has been completed.

3. The system of claim 1 , wherein the identification indication associated with the first item is different than the identification indication associated with the second item.

4. The system of claim 1 , wherein:

the identification indication associated with the first item includes a first highlight region positioned in connection with the first item;

the identification indication associated with the second item includes a second highlight region positioned in connection with the second item;

the first highlight region is associated with a first color to signify that the first item has the known identity; and

the second highlight region is associated with a second color to signify that the second item has the unknown identity.

5. The system of claim 1 , wherein:

the second item having the unknown identity includes the plurality of potential identities;

the identification indication associated with the first item includes a first highlight region positioned in connection with the first item;

the identification indication associated with the second item includes a second highlight region positioned in connection with the second item;

the first highlight region is associated with a first color to signify that the first item has the known identity; and

the second highlight region is associated with a second color to signify that the second item has the plurality of potential identities.

6. The system of claim 1 , wherein the hardware processor is further configured to:

receive a selection of the first item or the second item;

retrieve information relating to the selected item; and

provide display data, on a display device, in an overlay of the image of the content item based on the retrieved information, wherein the display data includes a link that directs the display device to supplemental information relating to the selected item.

7. The system of claim 1 , wherein the hardware processor is further configured to determine that the content item includes predetermined item identifier data within the content item received from a media source.

8. The system of claim 1 , wherein the hardware processor is further configured to identify a text region within the image by dividing the image into a plurality of regions, detecting features in each region of the plurality of regions, analyzing detected features in each region, and determining whether the detected features comprise textual information.

9. The system of claim 1 , wherein the hardware processor is further configured to present a number of the plurality of items displayed within the image concurrently with the image.

10. The system of claim 1 , wherein the identification indication for the unknown second item includes a prompt for user selection of an identity from the plurality of potential identities.

11. A method comprising:

receiving, using a hardware processor, an image from a content item;

determining, using the hardware processor, that the image includes a plurality of items displayed within the image;

transmitting, using the hardware processor, a plurality of search queries to one or more sources that search for known items that each correspond to one of the plurality of items displayed in the image while concurrently providing an initial search indication, for each of the plurality of items displayed in the image, indicating a search status for that item;

determining, using the hardware processor, based on the search, that the plurality of items displayed in the image includes a first item having a known identity and a second item having an unknown identity; and

causing, using the hardware processor, the image from the content item to be presented while providing an auditory feedback of the first item having the known identity and the second item having a plurality of potential identities and while providing an identification indication for each of the plurality of items displayed in the image, wherein the identification indication for the unknown second item includes a query to receive additional information that is used to identify the unknown second item.

12. The method of claim 11 , wherein the initial search indication is modified from an initial visual state indicating that the search for the corresponding known item is ongoing to a modified visual state indicating that the search for the corresponding known item has been completed.

13. The method of claim 11 , wherein the identification indication associated with the first item is different than the identification indication associated with the second item.

14. The method of claim 11 , wherein:

the identification indication associated with the first item includes a first highlight region positioned in connection with the first item;

the identification indication associated with the second item includes a second highlight region positioned in connection with the second item;

the first highlight region is associated with a first color to signify that the first item has the known identity; and

the second highlight region is associated with a second color to signify that the second item has the unknown identity.

15. The method of claim 11 , wherein:

the second item having the unknown identity includes the plurality of potential identities;

the identification indication associated with the first item includes a first highlight region positioned in connection with the first item;

the identification indication associated with the second item includes a second highlight region positioned in connection with the second item;

the first highlight region is associated with a first color to signify that the first item has the known identity; and

the second highlight region is associated with a second color to signify that the second item has the plurality of potential identities.

16. The method of claim 11 , further comprising:

receiving a selection of the first item or the second item;

retrieving information relating to the selected item; and

providing display data, on a display device, in an overlay of the image of the content item based on the retrieved information, wherein the display data includes a link that directs the display device to supplemental information relating to the selected item.

17. The method of claim 11 , further comprising determining that the content item includes predetermined item identifier data within the content item received from a media source.

18. The method of claim 11 , further comprising identifying a text region within the image by dividing the image into a plurality of regions, detecting features in each region of the plurality of regions, analyzing detected features in each region, and determining whether the detected features comprise textual information.

19. The method of claim 11 , further comprising presenting a number of the plurality of items displayed within the image concurrently with the image.

20. The method of claim 11 , wherein the identification indication for the unknown second item includes a prompt for user selection of an identity from the plurality of potential identities.

21. A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method, the method comprising:

receiving, using a hardware processor, an image from a content item;

determining, using the hardware processor, that the image includes a plurality of items displayed within the image;

transmitting, using the hardware processor, a plurality of search queries to one or more sources that search for known items that each correspond to one of the plurality of items displayed in the image while concurrently providing an initial search indication, for each of the plurality of items displayed in the image, indicating a search status for that item;

determining, using the hardware processor, based on the search, that the plurality of items displayed in the image includes a first item having a known identity and a second item having an unknown identity; and

causing, using the hardware processor, the image from the content item to be presented while providing an auditory feedback of the first item having the known identity and the second item having a plurality of potential identities and while providing an identification indication for each of the plurality of items displayed in the image, wherein the identification indication for the unknown second item includes a query to receive additional information that is used to identify the unknown second item.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2024
From: WITENSTEIN-WEAVER, JASON
To: GOOGLE INC.
Reel/Frame 067693/0562 →
ENTITY CONVERSION Recorded Jun 11, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 067698/0293 →
Continuity (4)
Continuation 16259197 · Jan 28, 2019
Continuation 15456157 · Mar 10, 2017
Continuation 13343630 · Jan 4, 2012
Related Publication 20230229690A1 · Jul 20, 2023