IP Library Granted Patent US 10,013,613
Granted Patent B2
US 10,013,613 · App. 15/077,942 · Granted Jul 3, 2018

Identifying objects in an image

Inventors: Nicolas E. Anastassacos (Heidelberg, DE); Chris R. Bean (Chandler's Ford, GB); Naresh Krishna Gopikrishnan (Southhampton, GB); Alexander Horsfield (Durham, GB); Joe Pavitt (Chandler's Ford, GB); Nicholas Wilkin (Newcastle Upon Tyne, GB)
Assignee: International Business Machines Corporation
G06K9/00691G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,013,613
App. No.
15/077,942
Granted
Jul 3, 2018
Kind
B2
Abstract

Identifying objects in an image. An image is received. A plurality of objects in the image are detected. One or more of the detected objects are identified, while one or more detected objects remain unidentified. From the identified detected objects, one or more salient objects are determined. Based on the salient objects, a generic location for the image is determined. Based on the determined generic location for the image, at least one of the unidentified detected objects is identified.

Claims (69)

1. A computer-implemented method for identifying objects in an image, the method comprising:

receiving an image;

detecting a plurality of objects in the image;

identifying one or more objects from the plurality of detected objects, wherein identifying comprises assigning a semantic label to at least one of the one or more detected objects, and wherein another one or more objects from the plurality of detected objects remain unidentified;

determining one or more salient objects from the one or more identified objects;

determining a generic location for the image, based on the one or more determined salient objects; and

identifying at least one of the one or more unidentified detected objects from the plurality of detected objects, based on the determined generic location for the image.

2. A method in accordance with claim 1 , wherein assigning a semantic label to a detected object comprises:

identifying a number of features of the detected object;

assigning a likelihood to the semantic label for the detected object, based on the identified features; and

identifying the particular object as corresponding to the semantic label if the assigned likelihood is greater than a predetermined threshold.

3. A method in accordance with claim 1 , wherein determining one or more salient objects from the one or more identified objects comprises applying one or more saliency algorithms to the one or more identified objects.

4. A method in accordance with claim 3 , wherein the saliency algorithms are selected from the group consisting of image segmentation and saliency maps.

5. A method in accordance with claim 1 , wherein determining a generic location for the image from the determined salient objects comprises:

accessing a first database defining a generic location likelihood for a list of specific objects, wherein a generic location likelihood corresponds to a probability that the image is from the generic location;

summing the generic location likelihoods for the determined salient objects; and

selecting a general location with a highest total of summed generic location likelihoods for the determined salient objects.

6. A method in accordance with claim 1 , wherein identifying at least one of the one or more unidentified detected objects from the plurality of detected objects, based on the determined generic location for the image, comprises:

accessing a second database defining a set of known objects for each of a list of generic locations;

increasing an assigned likelihood for the unidentified detected object if the unidentified detected object is present in the set of known objects for the determined generic location for the image; and

identifying the unidentified detected object if the increased assigned likelihood is greater than a predetermined threshold.

7. A method in accordance with claim 1 , further comprising:

providing an output specifying the determined generic location for the image, the identified detected objects, the identified unidentified detected objects, and an assigned likelihood for each of the identified detected objects and identified unidentified detected objects, the assigned likelihood defining a probability that the respective identified detected object or identified unidentified detected object is present in the image.

8. A computer system for identifying objects in an image, the computer system comprising:

one or more computer processors, one or more non-transitory computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising:

program instructions to receive an image;

program instructions to detect a plurality of objects in the image;

program instructions to identify one or more objects from the plurality of detected objects, wherein identifying comprises assigning a semantic label to at least one of the one or more detected objects, and wherein another one or more objects from the plurality of detected objects remain unidentified;

program instructions to determine one or more salient objects from the one or more identified objects;

program instructions to determine a generic location for the image, based on the one or more determined salient objects; and

program instructions to identify at least one of the one or more unidentified detected objects from the plurality of detected objects, based on the determined generic location for the image.

9. A computer system in accordance with claim 8 , wherein program instructions to assign a semantic label to a detected object comprise:

program instructions to identify a number of features of the detected object;

program instructions to assign a likelihood to the semantic label for the detected object, based on the identified features; and

program instructions to identify the particular object as corresponding to the semantic label if the assigned likelihood is greater than a predetermined threshold.

10. A computer system in accordance with claim 8 , wherein program instructions to determine one or more salient objects from the one or more identified objects comprise program instructions to apply one or more saliency algorithms to the one or more identified objects.

11. A computer system in accordance with claim 10 , wherein the saliency algorithms are selected from the group consisting of image segmentation and saliency maps.

12. A computer system in accordance with claim 8 , wherein program instructions to determine a generic location for the image from the determined salient objects comprise:

program instructions to access a first database defining a generic location likelihood for a list of specific objects, wherein a generic location likelihood corresponds to a probability that the image is from the generic location;

program instructions to sum the generic location likelihoods for the determined salient objects; and

program instructions to select a general location with a highest total of summed generic location likelihoods for the determined salient objects.

13. A computer system in accordance with claim 8 , wherein program instructions to identify at least one of the one or more unidentified detected objects from the plurality of detected objects, based on the determined generic location for the image, comprise:

program instructions to access a second database defining a set of known objects for each of a list of generic locations;

program instructions to increase an assigned likelihood for the unidentified detected object if the unidentified detected object is present in the set of known objects for the determined generic location for the image; and

program instructions to identify the unidentified detected object if the increased assigned likelihood is greater than a predetermined threshold.

14. A computer system in accordance with claim 8 , further comprising:

program instructions to provide an output specifying the determined generic location for the image, the identified detected objects, the identified unidentified detected objects, and an assigned likelihood for each of the identified detected objects and identified unidentified detected objects, the assigned likelihood defining a probability that the respective identified detected object or identified unidentified detected object is present in the image.

15. A computer program product for identifying objects in an image, the computer program product comprising:

one or more non-transitory computer-readable storage media and program instructions stored on the one or more computer-readable storage media, the program instructions comprising:

program instructions to receive an image;

program instructions to detect a plurality of objects in the image;

program instructions to identify one or more objects from the plurality of detected objects, wherein identifying comprises assigning a semantic label to at least one of the one or more detected objects, and wherein another one or more objects from the plurality of detected objects remain unidentified;

program instructions to determine one or more salient objects from the one or more identified objects;

program instructions to determine a generic location for the image, based on the one or more determined salient objects; and

program instructions to identify at least one of the one or more unidentified detected objects from the plurality of detected objects, based on the determined generic location for the image.

16. A computer program product in accordance with claim 15 , wherein program instructions to assign a semantic label to a detected object comprise:

program instructions to identify a number of features of the detected object;

program instructions to assign a likelihood to the semantic label for the detected object, based on the identified features; and

program instructions to identify the particular object as corresponding to the semantic label if the assigned likelihood is greater than a predetermined threshold.

17. A computer program product in accordance with claim 15 , wherein program instructions to determine one or more salient objects from the one or more identified objects comprise program instructions to apply one or more saliency algorithms to the one or more identified objects.

18. A computer program product in accordance with claim 17 , wherein the saliency algorithms are selected from the group consisting of image segmentation and saliency maps.

19. A computer program product in accordance with claim 15 , wherein program instructions to determine a generic location for the image from the determined salient objects comprise:

program instructions to access a first database defining a generic location likelihood for a list of specific objects, wherein a generic location likelihood corresponds to a probability that the image is from the generic location;

program instructions to sum the generic location likelihoods for the determined salient objects; and

program instructions to select a general location with a highest total of summed generic location likelihoods for the determined salient objects.

20. A computer program product in accordance with claim 15 , wherein program instructions to identify at least one of the one or more unidentified detected objects from the plurality of detected objects, based on the determined generic location for the image, comprise:

program instructions to access a second database defining a set of known objects for each of a list of generic locations;

program instructions to increase an assigned likelihood for the unidentified detected object if the unidentified detected object is present in the set of known objects for the determined generic location for the image; and

program instructions to identify the unidentified detected object if the increased assigned likelihood is greater than a predetermined threshold.

Assignments (7)
SECURITY AGREEMENT Recorded May 20, 2026
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 075591/0399 →
SECURITY INTEREST Recorded Nov 10, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 073514/0326 →
SECURITY AGREEMENT Recorded Mar 13, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 070513/0542 →
SECURITY AGREEMENT Recorded Oct 10, 2024
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 069143/0399 →
SECURITY AGREEMENT Recorded Mar 24, 2021
From: WAYFAIR LLC
To: CITIBANK, N.A.
Reel/Frame 055708/0832 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 29, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: WAYFAIR LLC
Reel/Frame 050867/0899 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2016
From: ANASTASSACOS, NICOLAS E.; BEAN, CHRIS R.; GOPIKRISHNAN, NARESH KRISHNA; HORSFIELD, ALEXANDER; PAVITT, JOE; WILKIN, NICHOLAS
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 038074/0118 →
Continuity (1)
Related Publication 20170278245A1 · Sep 28, 2017