IP Library Granted Patent US 9,442,933
Granted Patent B2
US 9,442,933 · App. 12/343,779 · Granted Sep 13, 2016

Identification of segments within audio, video, and multimedia items

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,442,933
App. No.
12/343,779
Granted
Sep 13, 2016
Kind
B2
Abstract

The invention pertains to methods, systems, and apparatus for identifying segments within a media item, the media segment including at least one of audio content and video content, comprising segmenting the media item into a plurality of segments as a function of subject matter, storing data identifying each segment and its subject matter, and organizing each segment within an ontology based on its subject matter.

Claims (72)

1. A method, comprising:

analyzing, to determine one or more keywords, descriptive data of a media item;

analyzing, by a computing device, the one or more keywords and selecting, based on the analyzing of the one or more keywords, a knowledge domain for the media item from a plurality of different knowledge domains;

identifying, based on the knowledge domain, a plurality of attribute fields associated with the knowledge domain;

identifying, from different combinations of media analysis technologies that correspond to the plurality of different knowledge domains and that are usable to analyze media, a combination of media analysis technologies corresponding to the knowledge domain;

analyzing, after identifying the combination of media analysis technologies, the media item using the combination of media analysis technologies to determine values for the plurality of attribute fields; and

segmenting the media item into a plurality of segments as a function of the values for the plurality of attribute fields by at least determining beginning and ending boundaries for the plurality of segments as a function of the values for the plurality of attribute fields.

2. The method of claim 1 , wherein the media item comprises at least one of audio content or video content, wherein the combination of media analysis technologies comprises a first media analysis technology, and wherein analyzing the media item using the combination of media analysis technologies to determine the values for the plurality of attribute fields comprises:

analyzing, using the first media analysis technology, the at least one of the audio content or the video content to determine one or more of the values for the plurality of attribute fields, wherein the plurality of attribute fields and the values for the plurality of attribute fields form a plurality of attribute/value pairs; and wherein the method further comprises:

associating the plurality of attribute/value pairs with at least one of the plurality of segments.

3. The method of claim 2 further comprising obtaining, from an information source different from a source of the media item, information about the media item; and

wherein segmenting the media item into the plurality of segments is performed as a function of the values for the plurality of attribute fields and the information about the media item.

4. The method of claim 1 further comprising:

classifying each of the plurality of segments within an ontology, resulting in one or more classifications of each segment; and

storing information identifying the plurality of segments to allow the plurality of segments to be found in future searches and storing information identifying the one or more classifications.

5. The method of claim 1 further comprising:

storing information identifying the plurality of segments to allow the plurality of segments to be found in future searches; and

performing, responsive to a search query, a search for segments based on the information identifying the plurality of segments.

6. The method of claim 1 wherein the media item comprises at least one of audio content or video content;

wherein the combination of media analysis technologies comprises at least one of a closed captioning analytics technology, speech recognition technology, audio analytics technology for analyzing non-speech aspects of audio of the audio content, optical character recognition technology, video analytics technology, or metadata analytics technology; and

wherein analyzing the media item using the combination of media analysis technologies to determine the values for the plurality of attribute fields comprises at least one of analyzing a closed captioning stream of the media item using the closed captioning analytics technology, analyzing the audio content of the media item for non-speech contextual information using the audio analytics technology, performing speech recognition on the audio content of the media item using the speech recognition technology, performing optical character recognition on the video content of the media item using the optical character recognition technology, performing video analytics on the video content of the media item using the video analytics technology, or analyzing metadata of the media item using the metadata analytics technology.

7. The method of claim 1 wherein the media item comprises multimedia assets of audio or video, and wherein the plurality of segments are sub-assets of the multimedia assets.

8. The method of claim 1 wherein the media item comprises television programming.

9. The method of claim 1 , wherein the plurality of attribute fields is unique to the knowledge domain.

10. The method of claim 1 , wherein the plurality of attribute fields comprises a set of knowledge domain specific media segmentation parameters that identify features to look for when segmenting media of the knowledge domain.

11. The method of claim 1 , further comprising:

storing the values for the plurality of attribute fields to a database; and

storing the beginning and the ending boundaries for the plurality of segments to the database.

12. A method, comprising:

storing a first set of attribute fields defined for a first knowledge domain;

storing a second set of attribute fields defined for a second knowledge domain different from the first knowledge domain;

storing data describing a first gathering process that is specific to the first knowledge domain and that identifies a first combination of media analysis technologies to use when segmenting media of the first knowledge domain;

storing, by a computing device, data describing a second gathering process that is specific to the second knowledge domain and that identifies a second combination of media analysis technologies to use when segmenting media of the second knowledge domain, the second combination of media analysis technologies being different from the first combination of media analysis technologies;

analyzing, to determine one or more keywords, descriptive data of a media item;

analyzing the one or more keywords and selecting, based on the analysis of the one or more keywords, the first knowledge domain;

identifying, after the first knowledge domain is selected, the first set of attribute fields;

retrieving the data describing the first gathering process;

analyzing, after retrieving the data describing the first gathering process, the media item using the first combination of media analysis technologies to determine values for the first set of attribute fields; and

segmenting the media item into a plurality of segments as a function of the first gathering process and the values for the first set of attribute fields by at least determining beginning and ending boundaries for the plurality of segments as a function of the values for the first set of attribute fields.

13. The method of claim 12 further comprising:

analyzing, to determine one or more second keywords, descriptive data of a second media item;

analyzing the one or more second keywords and selecting, based on the analysis of the one or more second keywords, the second knowledge domain;

identifying, after the second knowledge domain is selected, the second set of attribute fields;

retrieving the data describing the second gathering process;

analyzing, after retrieving the data describing the second gathering process, the second media item using the second combination of media analysis technologies to determine values for the second set of attribute fields; and

segmenting the second media item into a second plurality of segments as a function of the second gathering process and the values for the second set of attribute fields by at least determining beginning and ending boundaries for the second plurality of segments as a function of the values for the second set of attribute fields.

14. The method of claim 13 wherein the first knowledge domain is a first type of sport, and the second knowledge domain is a second type of sport.

15. The method of claim 14 wherein the first set of attribute fields comprises at least one of the following: a team on offense attribute field, a team on defense attribute field, a game time attribute field, a down number attribute field, a key offensive players attribute field, a key defensive player attribute field, a type of play attribute field, or a yards gained attribute field; and

wherein the second set of attribute fields comprises at least one of the following: an inning number attribute field, a batter attribute field, or a result of at-bat attribute field.

16. The method of claim 12 wherein the media item comprises at least one of audio content or video content;

wherein the first combination of media analysis technologies comprises at least one of a closed captioning analytics technology, speech recognition technology, audio analytics technology for analyzing non-speech aspects of audio of the audio content, optical character recognition technology, video analytics technology, or metadata analytics technology; and

wherein analyzing the media item using the first combination of media analysis technologies to determine the values for the first set of attribute fields comprises at least one of analyzing a closed captioning stream of the media item using the closed captioning analytics technology, analyzing the audio content for non-speech contextual information using the audio analytics technology, performing speech recognition on the audio content using speech recognition technology, performing optical character recognition on the video content using the optical character recognition technology, performing video analytics on the video content using the video analytics technology, or analyzing metadata of the media item using the metadata analytics technology.

17. The method of claim 12 , wherein the first combination of media analysis technologies comprises speech recognition technology, video analytics technology and metadata analytics technology, and

wherein the second combination of media analysis technologies comprises audio analytics technology for analyzing non-speech aspects of audio of the media item and optical character recognition technology.

18. The method of claim 12 , further comprising:

storing the values for the first set of attribute fields to a database; and

storing the beginning and the ending boundaries for the plurality of segments to the database.

19. A method, comprising:

analyzing, to determine one or more keywords, descriptive data of a media item, the media item comprising video content;

analyzing, by a computing device, the one or more keywords and selecting, based on the analyzing of the one or more keywords, a knowledge domain for the media item from a plurality of different knowledge domains;

identifying, based on the knowledge domain, a plurality of attribute fields associated with the knowledge domain;

selecting, from different pre-defined combinations of media analysis technologies corresponding to the plurality of different knowledge domains, a pre-defined combination of media analysis technologies that corresponds to the knowledge domain and that comprises a first media analysis technology for analyzing the video content;

analyzing, after selecting the pre-defined combination of media analysis technologies, the media item using the pre-defined combination of media analysis technologies to determine values for the plurality of attribute fields by at least analyzing the video content of the media item using the first media analysis technology to determine one or more first values of the values for the plurality of attribute fields; and

segmenting the media item into a plurality of segments as a function of the values for the plurality of attribute fields by at least determining beginning and ending boundaries for the plurality of segments as a function of the values for the plurality of attribute fields.

20. The method of claim 19 , wherein the media item comprises audio content;

wherein the pre-defined combination of media analysis technologies comprises a second media analysis technology for analyzing the audio content; and

wherein analyzing the media item using the pre-defined combination of media analysis technologies to determine the values for the plurality of attribute fields comprises analyzing the audio content using the second media analysis technology to determine one or more second values of the values for the plurality of attribute fields.

21. The method of claim 20 , wherein the second media analysis technology is software for analyzing the audio content to determine an identity of a person speaking within the audio content, and wherein the one or more second values comprises the identity of the person speaking within the audio content.

22. The method of claim 20 , wherein the second media analysis technology is software for analyzing the audio content to determine an occurrence of a particular type of sound within the audio content, and wherein the one or more second values comprises an indication of the occurrence of the particular type of sound.

23. The method of claim 19 , wherein the media item comprises metadata;

wherein the pre-defined combination of media analysis technologies comprises a second media analysis technology for analyzing the metadata; and

wherein analyzing the media item using the pre-defined combination of media analysis technologies to determine the values for the plurality of attribute fields comprises analyzing the metadata using the second media analysis technology to determine one or more second values of the values for the plurality of attribute fields.

Assignments (6)
CHANGE OF NAME Recorded Mar 31, 2026
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 075303/0717 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO CORPORATION
To: TIVO LLC
Reel/Frame 069083/0230 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO LLC
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 069083/0311 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2020
From: COMCAST INTERACTIVE MEDIA, LLC
To: TIVO CORPORATION
Reel/Frame 054540/0104 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2009
From: TZOUKERMANN, EVELYNE; CHIPMAN, LESLIE EUGENE; DAVIS, ANTHONY R.; HOUGHTON, DAVID F.; FARRELL, RYAN M.; ZHOU, HONGZHONG; JOJIC, OLIVER; SHEVADE, BAGESHREE; AMBWANI, GEETU; KRONROD, VLADIMIR
To: COMCAST INTERACTIVE MEDIA, LLC
Reel/Frame 022392/0765 →