IP Library Granted Patent US 10,896,355
Granted Patent B2
US 10,896,355 · App. 16/228,834 · Granted Jan 19, 2021

Automatic canonical digital image selection method and apparatus

Inventors: Sachin Sudhakar Farfade (Santa Clara, CA); Vijay Mahadevan (Santa Clara, CA); Ayman Kaheel (Denver, CO); Ayyappan Arasu (Cupertino, CA); Venkat Kumar Reddy Barakam (Cupertino, CA); Jay Kiran Mahadeokar (Santa Clara, CA)
Assignee: VERIZON MEDIA INC.
G06K9/6297G06F16/5854G06K9/00281G06K9/6224G06K9/6247G06K9/00228
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,896,355
App. No.
16/228,834
Granted
Jan 19, 2021
Kind
B2
Abstract

Disclosed are systems and methods for automatic selection of canonical digital images from a large corpus of digital images, such as the corpus of digital images available on the web, for an entity, such as and without limitation a person, a point of interest, object, etc. The automated, unsupervised approach for selecting a diverse set of high quality, canonical digital images, is well suited for processing a large corpus of digital images. A set of canonical digital images identified for an entity can be retrieved in response to a digital image request for digital images depicting the entity.

Claims (49)

1. A method comprising:

obtaining, via a computing device, a plurality of candidate digital images associated with an object, the object corresponding to an object type, a number of fiducial points being associated with the object type;

analyzing, via the computing device, each candidate digital image of the plurality, the analyzing comprising detecting the object and the number of fiducial points in the candidate digital image;

determining, via the computing device, an n-dimensional feature vector for a candidate digital image of the plurality using pixel data corresponding to the number of fiducial points detected in the candidate digital image, the feature vector determination being performed for each candidate digital image of the plurality to determine a plurality of feature vectors;

forming, via the computing device, a plurality of clusters using the plurality of feature vectors, each cluster of the plurality having a number of candidate digital images assigned to it using the feature vector determined for each candidate digital image of the number;

determining, via the computing device and for a cluster of the plurality of clusters, a canonical score for each candidate digital image assigned to the cluster;

selecting, via the computing device and for the cluster of the plurality of clusters, a set of canonical digital images from the number of candidate digital images assigned to the cluster; and

maintaining, via the computing device and in a data store, an association between a canonical digital image of the set, the canonical score determined for the canonical digital image and a set of search terms, the set of search terms being associated with the object, the maintaining being performed for each canonical digital image of the set such that each canonical digital image, of the set, has an associated canonical score and the associated set of search terms.

2. The method of claim 1 , further comprising:

receiving, via the computing device, a search request for digital images depicting the object, the search request comprising at least one search term from the set of search terms associated with each canonical digital image of the set; and

generating, via the computing device, a set of search results in response to the search request, the generating comprising determining that the at least one search term belongs to the set of search terms and selecting at least one canonical digital image from the set of canonical digital images associated with the set of search terms, the selecting using the associated canonical score for each canonical digital image of the set.

3. The method of claim 2 , the associated canonical score for each canonical digital image, of the set, is used to select a subset of the canonical digital images of the set for inclusion in the set of search results.

4. The method of claim 2 , generating the set of search results further comprising:

using the associated canonical score of each canonical digital image selected for inclusion in the set of search results in ranking each selected canonical digital image in the set of search results.

5. The method of claim 1 , the canonical score determined for a candidate digital image, assigned to the cluster, is a measure of quality of the candidate digital image.

6. The method of claim 5 , the measure of quality is determined based on a combination of measures, the measures comprising a first measure that is based on a determined proportion of the candidate digital image depicting the object, a second measure that is based on aesthetics of the candidate digital image and a third measure that is determined based on a location of the object in the candidate digital image.

7. The method of claim 6 , the first measure being determined based on a proportion of the pixels corresponding to the object detected in the candidate digital image relative to a total number of pixels of the candidate digital image.

8. The method of claim 6 , the third measure being determined based on the location of the object relative to a center location in the candidate digital image and the highest value assignable to the third measure is associated with the center location of the candidate digital image.

9. The method of claim 6 , the measure of quality that is determined based on the combination is a sum of the first, second and third measures.

10. The method of claim 6 , the measure of quality that is determined based on the combination is an average of the first, second and third measures.

11. The method of claim 1 , the selecting the set of canonical digital images from the number of candidate digital images assigned to the cluster comprising using the canonical score determined for each candidate digital image associated to the cluster to select a number of highest scoring candidate digital images for the set of canonical digital images.

12. The method of claim 1 , the set of search terms, associated with the object and associated with each canonical digital image depicting the object, being used in obtaining the plurality of candidate digital images.

13. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising:

obtaining a plurality of candidate digital images associated with an object, the object corresponding to an object type, a number of fiducial points being associated with the object type;

analyzing each candidate digital image of the plurality, the analyzing comprising detecting the object and the number of fiducial points in the candidate digital image;

determining an n-dimensional feature vector for a candidate digital image of the plurality using pixel data corresponding to the number of fiducial points detected in the candidate digital image, the feature vector determination being performed for each candidate digital image of the plurality to determine a plurality of feature vectors;

forming a plurality of clusters using the plurality of feature vectors, each cluster of the plurality having a number of candidate digital images assigned to it using the feature vector determined for each candidate digital image of the number;

determining, for a cluster of the plurality of clusters, a canonical score for each candidate digital image assigned to the cluster;

selecting, for the cluster of the plurality of clusters, a set of canonical digital images from the number of candidate digital images assigned to the cluster; and

maintaining, in a data store, an association between a canonical digital image of the set, the canonical score determined for the canonical digital image and a set of search terms, the set of search terms being associated with the object, the maintaining being performed for each canonical digital image of the set such that each canonical digital image, of the set, has an associated canonical score and the associated set of search terms.

14. The non-transitory computer-readable storage medium of claim 13 , the method further comprising:

receiving a search request for digital images depicting the object, the search request comprising at least one search term from the set of search terms associated with each canonical digital image of the set; and

generating a set of search results in response to the search request, the generating comprising determining that the at least one search term belongs to the set of search terms and selecting at least one canonical digital image from the set of canonical digital images associated with the set of search terms, the selecting using the associated canonical score for each canonical digital image of the set.

15. The non-transitory computer-readable storage medium of claim 14 , the associated canonical score for each canonical digital image, of the set, is used to select a subset of the canonical digital images of the set for inclusion in the set of search results.

16. The non-transitory computer-readable storage medium of claim 14 , generating the set of search results further comprising:

using the associated canonical score of each canonical digital image selected for inclusion in the set of search results in ranking each selected canonical digital image in the set of search results.

17. The non-transitory computer-readable storage medium of claim 13 , the canonical score determined for a candidate digital image, assigned to the cluster, is a measure of quality of the candidate digital image.

18. The non-transitory computer-readable storage medium of claim 17 , the measure of quality is determined based on a combination of measures, the measures comprising a first measure that is based on a determined proportion of the candidate digital image depicting the object, a second measure that is based on aesthetics of the candidate digital image and a third measure that is determined based on a location of the object in the candidate digital image.

19. The transitory computer-readable storage medium of 33 , the set of search terms, associated with the object and associated with each canonical digital image depicting the object, being used in obtaining the plurality of candidate digital images.

20. A computing device comprising:

a processor;

a non-transitory storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

obtaining logic executed by the processor for obtaining a plurality of candidate digital images associated with an object, the object corresponding to an object type, a number of fiducial points being associated with the object type;

analyzing logic executed by the processor for analyzing each candidate digital image of the plurality, the analyzing comprising detecting the object and the number of fiducial points in the candidate digital image;

determining logic executed by the processor for determining an n-dimensional feature vector for a candidate digital image of the plurality using pixel data corresponding to the number of fiducial points detected in the candidate digital image, the feature vector determination being performed for each candidate digital image of the plurality to determine a plurality of feature vectors;

forming logic executed by the processor for forming a plurality of clusters using the plurality of feature vectors, each cluster of the plurality having a number of candidate digital images assigned to it using the feature vector determined for each candidate digital image of the number;

determining logic executed by the processor for determining, for a cluster of the plurality of clusters, a canonical score for each candidate digital image assigned to the cluster;

selecting logic executed by the processor for selecting, for the cluster of the plurality of clusters, a set of canonical digital images from the number of candidate digital images assigned to the cluster; and

maintaining logic executed by the processor for maintaining, in a data store, an association between a canonical digital image of the set, the canonical score determined for the canonical digital image and a set of search terms, the set of search terms being associated with the object, the maintaining being performed for each canonical digital image of the set such that each canonical digital image, of the set, has an associated canonical score and the associated set of search terms.

Assignments (6)
PATENT SECURITY AGREEMENT (FIRST LIEN) Recorded Sep 29, 2022
From: YAHOO ASSETS LLC
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 061571/0773 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2021
From: YAHOO AD TECH LLC (FORMERLY VERIZON MEDIA INC.)
To: YAHOO ASSETS LLC
Reel/Frame 058982/0282 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: OATH INC.
To: VERIZON MEDIA INC.
Reel/Frame 054258/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2019
From: FARFADE, SACHIN SUDHAKAR; MAHADEVAN, VIJAY; KAHEEL, AYMAN; ARASU, AYYAPPAN; BARAKAM, VENKAT KUMAR REDDY; MAHADEOKAR, JAY KIRAN
To: YAHOO! INC.
Reel/Frame 048439/0480 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2019
From: YAHOO! INC.
To: YAHOO HOLDINGS, INC.
Reel/Frame 048441/0310 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2019
From: YAHOO HOLDINGS, INC.
To: OATH INC.
Reel/Frame 048442/0001 →
Continuity (2)
Continuation 15198295 · Jun 30, 2016
Related Publication 20190147301A1 · May 16, 2019