IP Library Granted Patent US 9,009,163
Granted Patent B2
US 9,009,163 · App. 12/633,266 · Granted Apr 14, 2015

Lazy evaluation of semantic indexing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,009,163
App. No.
12/633,266
Granted
Apr 14, 2015
Kind
B2
Abstract

A method for searching a database of digital media assets that has been indexed according to a set of general indexers. The method includes receiving a search query and defining specialized search conditions by identifying one or more elements of the search query corresponding to one or more specialized indexers. General search conditions are defined by identifying elements of the search query corresponding to the general indexers. A subset of the digital media assets are identified by applying the general search conditions, and the subset of the digital media assets is indexed using the identified specialized indexers. The subset of the digital media assets are ranked by applying the specialized search conditions.

Claims (58)

1. A method for searching a database of digital media assets, comprising:

a processor receiving a search query;

the processor analyzing the search query to extract a specialized search condition corresponding to a specialized semantic concept;

the processor selecting, from a pre-existing ontology of indexers, a specialized indexer based on a determination that the selected specialized indexer can be used to identify digital media assets that correspond to the specialized semantic concept;

the processor analyzing the search query to extract a general search condition corresponding to a related higher level semantic concept;

the processor identifying, from a database of digital media assets, a subset of digital media assets that satisfy the general search condition responsive to general metadata for each of the digital media assets;

the processor applying the selected specialized indexer to the subset of digital media assets to determine specialized metadata for a digital media asset in the subset of digital media assets, wherein the specialized metadata provides an indication of whether the digital media asset corresponds to the specialized semantic concept; and

the processor converting the specialized indexer into a general indexer based at least on a frequency with which the specialized semantic concept is queried, wherein the processor is configured to convert the specialized indexer into the general indexer based further on a computational cost associated with the specialized indexer,

wherein the general indexer is applied to the digital media assets independent of a search query, wherein the general indexer is operable to identify a digital media asset that corresponds to a higher level semantic concept and determine the general metadata for the identified digital media asset based at least upon an analysis of pixel data associated with the digital media asset, and wherein the general metadata provides an indication of the higher level semantic concept to which the identified digital media asset corresponds.

2. The method of claim 1 further comprising ranking the subset of the digital media assets according to their satisfaction of the general search condition responsive to the general metadata.

3. The method of claim 1 wherein an element of the search query corresponds to both a general indexer and a specialized indexer.

4. The method of claim 1 wherein the specialized metadata is associated with the digital media assets and stored in the database of digital media assets.

5. The method of claim 1 wherein the specialized metadata already exists for a portion of the subset of the digital media assets, and wherein the specialized indexer is only applied to digital media assets not having the specialized metadata.

6. The method of claim 1 wherein the search query is received from a user.

7. The method of claim 1 wherein general search conditions are identified from elements of the search query by using concept expansion.

8. The method of claim 1 wherein specialized search conditions are identified from elements of the search query by using concept expansion.

9. The method of claim 6 further comprising:

ranking the subset of the digital media assets according to their satisfaction of the specialized search condition responsive to the specialized metadata; and

presenting the most highly ranked set of digital media assets.

10. The method of claim 9 wherein the digital media assets are digital images and the digital media assets are presented by displaying the digital images.

11. The method of claim 9 wherein the digital media assets are digital video files and the digital media assets are presented by playing the digital video files.

12. The method of claim 9 wherein the digital media assets are digital audio files and the digital media assets are presented by playing the digital audio files.

13. The method of claim 9 wherein the digital media assets are complex objects composed of a plurality of digital images, digital videos or digital audios.

14. The method of claim 1 wherein the general metadata includes date/time metadata, event classification metadata, indoor/outdoor metadata, person detection metadata or geospatial metadata.

15. The method of claim 1 wherein the specialized metadata is a meronym of a generalized metadata term.

16. The method of claim 1 wherein a portion of the general metadata or the specialized metadata are specified by probability values.

17. The method of claim 16 further comprising using the probability values to rank the subset of the digital media assets.

18. The method of claim 1 further comprising presenting a query requesting additional information to clarify the search query.

19. The method of claim 18 wherein the query requests clarification of ambiguous elements of the search query.

20. The method of claim 1 wherein the general indexer is included in a set of general indexers, and wherein the set of general indexers is modified over time in response to a frequency at which search queries are executed containing elements corresponding to each indexer in the set of general indexers.

21. The method of claim 1 , wherein specialized search indexers are applied to digital media assets only in response to particular search queries.

22. The method of claim 1 , further comprising:

ranking the subset of the digital media assets according to their satisfaction of the general search condition based on the general metadata; and

ranking the subset of the digital media assets according to their satisfaction of the specialized search condition based on the specialized metadata.

23. The method of claim 1 , wherein analyzing the search query to extract a general search condition comprises selecting general indexers from the pre-existing ontology, and wherein the pre-existing ontology relates the general indexers to the selected specialized indexers.

24. The method of claim 1 , further comprising selecting a second specialized indexer based on a determination that the pre-existing ontology relates the second specialized indexer to the selected specialized indexer.

25. The method of claim 1 , wherein the pre-existing ontology comprises user-specific ontological relationships.

26. A system comprising:

a data processor; and

a memory communicatively connected to the data processor and storing instructions configured to cause the data processor to perform operations comprising:

receiving a search query;

analyzing the search query to extract a specialized search condition corresponding to a specialized semantic concept;

selecting, from a pre-existing ontology of indexers, a specialized indexer based on a determination that the selected specialized indexer can be used to identify digital media assets that correspond to the specialized semantic concept;

analyzing the search query to extract a general search condition corresponding to a related higher level semantic concept;

identifying, from a database of digital media assets, a subset of digital media assets that satisfy the general search condition responsive to general metadata for each of the digital media assets;

applying the selected specialized indexer to the subset of digital media assets to determine specialized metadata for a digital media asset in the subset of digital media assets, wherein the specialized metadata provides an indication of whether the digital media asset corresponds to the specialized semantic concept; and

the processor converting the specialized indexer into a general indexer based at least on a frequency with which the specialized semantic concept is queried, wherein the processor is configured to convert the specialized indexer into the general indexer based further on a computational cost associated with the specialized indexer;

wherein the general indexer is applied to the digital media assets independent of a search query, wherein the general indexer is operable to identify a digital media asset that corresponds to a higher level semantic concept and determine the general metadata for the identified digital media asset based at least upon an analysis of pixel data associated with the digital media asset, and wherein the general metadata provides an indication of the higher level semantic concept to which the identified digital media asset corresponds.

27. A non-transitory computer readable medium having stored thereon instructions executable by a processor to cause the processor to perform functions comprising:

receiving a search query;

analyzing the search query to extract a specialized search condition corresponding to a specialized semantic concept;

selecting, from a pre-existing ontology of indexers, a specialized indexer based on a determination that the selected specialized indexer can be used to identify digital media assets that correspond to the specialized semantic concept;

analyzing the search query to extract a general search condition corresponding to a related higher level semantic concept;

identifying, from a database of digital media assets, a subset of digital media assets that satisfy the general search condition responsive to general metadata for each of the digital media assets;

applying the selected specialized indexer to the subset of digital media assets to determine specialized metadata for a digital media asset in the subset of digital media assets, wherein the specialized metadata provides an indication of whether the digital media asset corresponds to the specialized semantic concept; and

converting the specialized indexer into a general indexer based at least on a frequency with which the specialized semantic concept is queried,

wherein the processor is configured to convert the specialized indexer into the general indexer based further on a computational cost associated with the specialized indexer,

wherein the general indexer is applied to the digital media assets independent of a search query, wherein the general indexer is operable to identify a digital media asset that corresponds to a higher level semantic concept and determine the general metadata for the identified digital media asset based at least upon an analysis of pixel data associated with the digital media asset, and wherein the general metadata provides an indication of the higher level semantic concept to which the identified digital media asset corresponds.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Aug 15, 2023
From: INTELLECTUAL VENTURES FUND 83 LLC
To: MONUMENT PEAK VENTURES, LLC
Reel/Frame 064599/0304 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2017
From: INTELLECTUAL VENTURES FUND 83 LLC
To: MONUMENT PEAK VENTURES, LLC
Reel/Frame 041941/0079 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 11, 2013
From: EASTMAN KODAK COMPANY
To: INTELLECTUAL VENTURES FUND 83 LLC
Reel/Frame 029959/0085 →
PATENT RELEASE Recorded Feb 1, 2013
From: CITICORP NORTH AMERICA, INC.; WILMINGTON TRUST, NATIONAL ASSOCIATION
To: EASTMAN KODAK COMPANY; EASTMAN KODAK INTERNATIONAL CAPITAL COMPANY, INC.; FAR EAST DEVELOPMENT LTD.; KODAK (NEAR EAST), INC.; KODAK AMERICAS, LTD.; KODAK PORTUGUESA LIMITED; KODAK REALTY, INC.; LASER-PACIFIC MEDIA CORPORATION; KODAK AVIATION LEASING LLC; KODAK PHILIPPINES, LTD.; NPEC INC.; FPC INC.; KODAK IMAGING NETWORK, INC.; PAKON, INC.; QUALEX INC.; CREO MANUFACTURING AMERICA LLC
Reel/Frame 029913/0001 →
SECURITY INTEREST Recorded Feb 21, 2012
From: EASTMAN KODAK COMPANY; PAKON, INC.
To: CITICORP NORTH AMERICA, INC., AS AGENT
Reel/Frame 028201/0420 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2010
From: HIBINO, STACIE L.; WOOD, MARK D.
To: EASTMAN KODAK
Reel/Frame 023801/0887 →