IP Library Granted Patent US 10,748,007
Granted Patent B2
US 10,748,007 · App. 15/943,818 · Granted Aug 18, 2020

Identifying objects in an image

Inventors: Nicolas E. Anastassacos (Heidelberg, DE); Chris R. Bean (Chandler's Ford, GB); Naresh Krishna Gopikrishnan (Southhampton, GB); Alexander Horsfield (Durham, GB); Joe Pavitt (Chandler's Ford, GB); Nicholas Wilkin (Newcastle upon Tyne, GB)
Assignee: Wayfair LLC
G06K9/00691G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,748,007
App. No.
15/943,818
Granted
Aug 18, 2020
Kind
B2
Abstract

Identifying objects in an image. An image is received. One or more objects in the image are identified, based on a database of identified objects, and wherein one or more other objects in the image are unidentified based on the database of identified objects. One or more salient objects in the image is identified, based on execution of a saliency algorithm. A generic location for the image is determined, based on the one or more identified salient objects and a database that associates objects with generic locations. One or more of the unidentified objects are identified, based on the determined generic location for the image.

Claims (64)

1. A method for identifying objects in an image, the method comprising:

receiving, by a computer, an image;

identifying, by the computer, one or more first objects in the image based on information in a first database of objects, wherein one or more second objects in the image are unidentified based on the information in the first database of objects;

identifying, by the computer, one or more salient objects in the image based on execution of a saliency algorithm;

determining, by the computer, a generic location for the image, based on the one or more identified salient objects and information in a second database that associates objects with generic locations; and

identifying, by the computer, the one or more second objects, based on the determined generic location for the image, wherein identifying the one or more second objects comprises:

accessing, by the computer, a third database that defines a set of known objects for each of a list of generic locations;

increasing, by the computer, an assigned likelihood for a second object of the one or more second objects if the second object is in the set of known objects for the determined generic location for the image; and

identifying, by the computer, the second object if the increased assigned likelihood is greater than a threshold.

2. A method in accordance with claim 1 , wherein identifying an object of the one or more first objects in the image comprises:

identifying, by the computer, features of the object;

identifying the object if a number of identified features of the object is above a predetermined threshold value; and

assigning a likelihood to a semantic label for the object based on the identified features.

3. A method in accordance with claim 1 , wherein the saliency algorithm is selected from the group consisting of image segmentation and saliency maps.

4. A method in accordance with claim 1 , wherein determining a generic location for the image based on the identified salient objects comprises:

accessing, by the computer, the second database that associates objects with generic locations, wherein the generic locations in the second database includes generic location likelihoods for the identified salient objects;

summing, by the computer, the generic location likelihoods for the identified salient objects; and

selecting, by the computer, the general location with a highest total of summed generic location likelihoods for the identified salient objects.

5. A method in accordance with claim 1 , further comprising:

providing, by the computer, an output specifying the determined generic location for the image, the identified first objects, the identified second objects, and an assigned likelihood for each of the identified first objects and identified second objects, the assigned likelihood defining a probability that the respective identified first object or identified second object is present in the image.

6. A computer system for identifying objects in an image, the computer system comprising:

one or more processors;

one or more computer readable storage media; and

program instructions stored on at least one of the one or more computer readable storage media, which when executed cause at least one of the one or more processors to perform a method comprising:

receiving, by a computer, an image;

identifying, by the computer, one or more first objects in the image based on information in a first database of objects, wherein one or more second objects in the image are unidentified based on the information in the first database of objects;

identifying, by the computer, one or more salient objects in the image based on execution of a saliency algorithm;

determining, by the computer, a generic location for the image, based on the one or more identified salient objects and information in a second database that associates objects with generic locations; and

identifying, by the computer, the one or more second objects, based on the determined generic location for the image, wherein identifying the one or more second objects comprises:

accessing, by the computer, a third database that defines a set of known objects for each of a list of generic locations;

increasing, by the computer, an assigned likelihood for a second object of the one or more second objects if the second object is in the set of known objects for the determined generic location for the image; and

identifying, by the computer, the second object if the increased assigned likelihood is greater than a threshold.

7. A computer system in accordance with claim 6 , wherein identifying an object of the one or more first objects in the image comprises:

identifying, by the computer, features of the object;

identifying the object if a number of identified features of the object is above a predetermined threshold value; and

assigning a likelihood to a semantic label for the object based on the identified features.

8. A computer system in accordance with claim 6 , wherein the saliency algorithm is selected from the group consisting of image segmentation and saliency maps.

9. A computer system in accordance with claim 6 , wherein determining a generic location for the image based on the identified salient objects comprises:

accessing, by the computer, the second database that associates objects with generic locations, wherein the generic locations in the second database includes generic location likelihoods for the identified salient objects;

summing, by the computer, the generic location likelihoods for the identified salient objects; and

selecting, by the computer, the general location with a highest total of summed generic location likelihoods for the identified salient objects.

10. A computer system in accordance with claim 6 , wherein the method further comprises:

providing, by the computer, an output specifying the determined generic location for the image, the identified first objects, the identified second objects, and an assigned likelihood for each of the identified first objects and identified second objects, the assigned likelihood defining a probability that the respective identified first object or identified second object is present in the image.

11. A computer program product for identifying objects in an image, the computer program product comprising:

one or more non-transitory computer readable storage media and program instructions stored on at least one of the one or more non-transitory computer readable storage media, the program instructions, when executed by a computer, cause the computer to perform a method comprising:

receiving, by a computer, an image;

identifying, by the computer, one or more first objects in the image based on information in a first database of objects, wherein one or more second objects in the image are unidentified based on the information in the first database of objects;

identifying, by the computer, one or more salient objects in the image based on execution of a saliency algorithm;

determining, by the computer, a generic location for the image, based on the one or more identified salient objects and information in a second database that associates objects with generic locations; and

identifying, by the computer, the one or more second objects, based on the determined generic location for the image, wherein identifying the one or more second objects comprises:

accessing, by the computer, a third database that defines a set of known objects for each of a list of generic locations;

increasing, by the computer, an assigned likelihood for a second object of the one or more second objects if the second object is in the set of known objects for the determined generic location for the image; and

identifying, by the computer, the second object if the increased assigned likelihood is greater than a threshold.

12. A computer program product in accordance with claim 11 , wherein identifying an object of the one or more first objects in the image comprises:

identifying, by the computer, features of the object;

identifying the object if a number of identified features of the object is above a predetermined threshold value; and

assigning a likelihood to a semantic label for the object based on the identified features.

13. A computer program product in accordance with claim 11 , wherein the saliency algorithm is selected from the group consisting of image segmentation and saliency maps.

14. A computer program product in accordance with claim 11 , wherein determining a generic location for the image based on the identified salient objects comprises:

accessing, by the computer, the second database that associates objects with generic locations, wherein the generic locations in the second database includes generic location likelihoods for the identified salient objects;

summing, by the computer, the generic location likelihoods for the identified salient objects; and

selecting, by the computer, the general location with a highest total of summed generic location likelihoods for the identified salient objects.

15. A computer program product in accordance with claim 11 , wherein the method further comprises:

providing, by the computer, an output specifying the determined generic location for the image, the identified first objects, the identified second objects, and an assigned likelihood for each of the identified first objects and identified second objects, the assigned likelihood defining a probability that the respective identified first object or identified second object is present in the image.

Assignments (7)
SECURITY AGREEMENT Recorded May 20, 2026
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 075591/0399 →
SECURITY INTEREST Recorded Nov 10, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 073514/0326 →
SECURITY AGREEMENT Recorded Mar 13, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 070513/0542 →
SECURITY AGREEMENT Recorded Oct 10, 2024
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 069143/0399 →
SECURITY AGREEMENT Recorded Mar 24, 2021
From: WAYFAIR LLC
To: CITIBANK, N.A.
Reel/Frame 055708/0832 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 29, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: WAYFAIR LLC
Reel/Frame 050867/0899 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2018
From: ANASTASSACOS, NICOLAS E.; BEAN, CHRIS R.; GOPIKRISHNAN, NARESH KRISHNA; HORSFIELD, ALEXANDER; PAVITT, JOE; WILKIN, NICHOLAS
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 045421/0501 →
Continuity (2)
Continuation 15077942 · Mar 23, 2016
Related Publication 20180225514A1 · Aug 9, 2018