IP Library Granted Patent US 8,625,904
Granted Patent B2
US 8,625,904 · App. 13/221,078 · Granted Jan 7, 2014

Detecting recurring themes in consumer image collections

Inventors: Madirakshi Das (Penfield, NY); Alexander C. Loui (Penfield, NY)
Assignee: Intellectual Ventures Fund 83 LLC
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,625,904
App. No.
13/221,078
Granted
Jan 7, 2014
Kind
B2
Abstract

A method of identifying groups of related digital images in a digital image collection, comprising: analyzing each of the digital images to generate associated feature descriptors related to image content or image capture conditions; storing the feature descriptors associated with the digital images in a metadata database; automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images; determining a probability of occurrence for each the identified frequent itemsets; determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence; ranking the frequent itemsets based at least on the determined quality scores; and identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets.

Claims (51)

1. A method comprising:

analyzing each digital image in a digital image collection to generate associated feature descriptors related to image content or image capture conditions;

storing the feature descriptors associated with the digital images in a metadata database;

using a data processor to automatically analyze the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a set of co-occurring feature descriptors that occurs in at least a predefined fraction of the digital images, each frequent itemset being associated with a subset of the digital images;

determining, for each of the identified frequent itemsets, a probability of occurrence representing a probability that a respective frequent itemset occurs in a general population of images based on one or more probability distributions determined from an analysis of a large number of image collections;

determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence;

ranking the frequent itemsets based at least on the determined quality scores;

identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets; and

storing an indication of the identified groups of related digital images in a processor-accessible memory.

2. The method of claim 1 , further comprising:

automatically analyzing the digital images using an event clustering operation to cluster the digital images into a set of events;

determining a confidence score for each of the frequent itemsets based at least upon the events associated with the digital images corresponding to the frequent itemset;

identifying any frequent itemsets having an associated confidence score lower than a pre-defined threshold; and

removing the identified frequent itemsets from the plurality of frequent itemsets.

3. The method of claim 2 wherein the confidence score for an itemset is based on a comparison of the number of images from a given event that are associated with the itemset to the total number of images from the event.

4. The method of claim 1 wherein the metadata database is stored using a transactional database format.

5. The method of claim 1 wherein the feature descriptors generated for the digital images include feature descriptors determined by analyzing image content of the digital images.

6. The method of claim 5 wherein the feature descriptors determined by analyzing image content of the digital images include scene class features, materials class features, low-level features or facial features.

7. The method of claim 1 wherein the feature descriptors generated for the digital images include feature descriptors determined by analyzing capture metadata associated with the digital images.

8. The method of claim 7 wherein the feature descriptors determined by analyzing capture metadata include capture condition features.

9. The method of claim 1 wherein the feature descriptors generated for the digital images include feature descriptors corresponding to metadata tags associated with the digital images.

10. The method of claim 9 wherein the metadata tags are received via a user-interface.

11. The method of claim 9 wherein the metadata tags were determined by automatically analyzing image content or capture metadata associated with the digital images.

12. The method of claim 1 wherein the feature descriptors are quantized into a finite number of predefined states.

13. The method of claim 1 wherein the ranking of the frequent itemsets is also based upon a time difference between an earliest capture time and a latest capture time associated with the digital images corresponding to the frequent itemset.

14. The method of claim 1 wherein the ranking of the frequent itemsets is also based upon the number of people in the digital images corresponding to the frequent itemset.

15. The method of claim 1 wherein the detected related images are displayed to a user in the form of a slideshow or photo-book.

16. A system comprising:

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method comprising:

analyzing each digital image in a digital image collection to generate associated feature descriptors related to image content or image capture conditions;

storing the feature descriptors associated with the digital images in a metadata database;

automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a set of co-occurring feature descriptors that occurs in at least a predefined fraction of the digital images, each frequent itemset being associated with a subset of the digital images;

determining, for each of the identified frequent itemsets, a probability of occurrence representing a probability that a respective frequent itemset occurs in a general population of images based on one or more probability distributions determined from an analysis of a large number of image collections;

determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence;

ranking the frequent itemsets based at least on the determined quality scores;

identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets; and

storing an indication of the identified groups of related digital images in a processor-accessible memory.

17. The system of claim 16 , wherein the confidence score for an itemset is determined based at least in part on the number of images from the same events in the digital images corresponding to the frequent itemset.

18. The system of claim 16 , wherein the feature descriptors determined by analyzing image content of the digital images include scene class features, materials class features, low-level features and facial features.

19. The system of claim 16 , wherein the metadata tags are received via a user-interface.

20. A non-transitory computer readable medium having stored thereon instructions executable by a processor to cause the processor to perform functions, comprising:

analyzing each digital image in a digital image collection to generate associated feature descriptors related to image content or image capture conditions;

storing the feature descriptors associated with the digital images in a metadata database;

automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring group of feature descriptors that occurs in at least a predefined fraction of the digital images, each frequent itemset being associated with a subset of the digital images;

determining, for each of the identified frequent itemsets, a probability of occurrence representing a probability that a respective frequent itemset occurs in a general population of images based on one or more probability distributions determined from an analysis of a large number of image collections;

determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence;

ranking the frequent itemsets based at least on the determined quality scores;

identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets; and

storing an indication of the identified groups of related digital images in a processor-accessible memory.

21. The method of claim 1 , wherein the quality score of a frequent itemset is inversely related to a commonness of the co-occurring feature descriptors in the frequent itemset.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Aug 15, 2023
From: INTELLECTUAL VENTURES FUND 83 LLC
To: MONUMENT PEAK VENTURES, LLC
Reel/Frame 064599/0304 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2017
From: INTELLECTUAL VENTURES FUND 83 LLC
To: MONUMENT PEAK VENTURES, LLC
Reel/Frame 041941/0079 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 12, 2013
From: EASTMAN KODAK COMPANY
To: INTELLECTUAL VENTURES FUND 83 LLC
Reel/Frame 029969/0477 →
PATENT RELEASE Recorded Feb 1, 2013
From: CITICORP NORTH AMERICA, INC.; WILMINGTON TRUST, NATIONAL ASSOCIATION
To: EASTMAN KODAK COMPANY; EASTMAN KODAK INTERNATIONAL CAPITAL COMPANY, INC.; FAR EAST DEVELOPMENT LTD.; KODAK (NEAR EAST), INC.; KODAK AMERICAS, LTD.; KODAK PORTUGUESA LIMITED; KODAK REALTY, INC.; LASER-PACIFIC MEDIA CORPORATION; KODAK AVIATION LEASING LLC; KODAK PHILIPPINES, LTD.; NPEC INC.; FPC INC.; KODAK IMAGING NETWORK, INC.; PAKON, INC.; QUALEX INC.; CREO MANUFACTURING AMERICA LLC
Reel/Frame 029913/0001 →
SECURITY INTEREST Recorded Feb 21, 2012
From: EASTMAN KODAK COMPANY; PAKON, INC.
To: CITICORP NORTH AMERICA, INC., AS AGENT
Reel/Frame 028201/0420 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2011
From: DAS, MADIRAKSHI; LOUI, ALEXANDER C.
To: EASTMAN KODAK
Reel/Frame 026828/0309 →
Continuity (1)
Related Publication 20130051670A1 · Feb 28, 2013