Content identification system
View Patent ↗The content of a media program is recognized by analyzing its audio content to extract therefrom prescribed features, which are compared to a database of features associated with identified content. The identity of the content within the database that has features that most closely match the features of the media program being played is supplied as the identity of the program being played. The features are extracted from a frequency domain version of the media program by a) filtering the coefficients to reduce their number, e.g., using triangular filters; b) grouping a number of consecutive outputs of triangular filters into segments; and c) selecting those segments that meet prescribed criteria, such as those segments that have the largest minimum segment energy with prescribed constraints that prevent the segments from being too close to each other. The triangular filters may be log-spaced and their output may be normalized.
1. A method for use in recognizing the content of a media program, said method comprising the steps of:
filtering each first frequency domain representation of blocks of said media program using a plurality of filters to develop a respective second frequency domain representation of each of said blocks of said media program, said second frequency domain representation of each of said blocks having a reduced number of frequency coefficients with respect to said first frequency domain representation;
grouping frequency coefficients of said second frequency domain representation of said blocks to form frequency coefficient segments;
selecting a plurality of said segments as representing said media program;
comparing said selected segments to frequency coefficient segments of stored programs to determine thereby corresponding matching scores; and
identifying said media program using said matching scores,
wherein said first frequency domain representation of blocks of said media program is developed by:
digitizing an audio representation of said media program;
dividing the digitized audio representation into time domain blocks of a prescribed number of samples;
smoothing said time domain blocks using a filter; and
converting said smoothed time domain blocks into frequency domain blocks, wherein said smoothed time domain blocks are represented by frequency coefficients.
2. The method of claim 1 , wherein each grouping of frequency coefficients of said second frequency domain to form a segment represents blocks that are consecutive in time in said media program.
3. The method of claim 1 , wherein said plurality of filters are arranged in a group that processes a block at a time, the portion of said second frequency domain representation produced by said group of filters for each block forms a frame, and wherein at least two frames are grouped to form a segment.
4. The method of claim 1 , wherein said selected segments correspond to portions of said media program that are not contiguous in time.
5. The method of claim 1 , wherein said plurality of filters includes at least a set of triangular filters.
6. The method of claim 1 , wherein said plurality of filters includes at least a set of log-spaced triangular filters.
7. The method of claim 1 , wherein the selected segments are those that have largest minimum segment energy.
8. The method of claim 1 , wherein the selected segments are selected in accordance with prescribed constraints such that the selected segments are separated by a minimum distance.
9. The method of claim 1 , wherein the selected segments are selected to contain portions of said media program that correspond in time to prescribed search windows that are separated by gaps.
10. The method of claim 1 , wherein the selected segments are those having a maximum entropy.
11. The method of claim 1 , further comprising normalizing frequency coefficients forming said selected segments.
12. The method of claim 11 , wherein said normalizing includes performing at least preceding-time normalizing of said selected segments.
13. The method of claim 11 , wherein said normalizing includes performing at least an L2 normalization.
14. The method of claim 1 , further comprising storing said selected segments in a database in association with an identifier of said media program.
15. The method of claim 14 , further comprising storing in said database timing information associated with said selected segments.
16. The method of claim 1 , further comprising indicating that said media program cannot be identified when matching scores above a sufficient correlation level are not determined.
17. The method of claim 1 , wherein:
said frequency coefficient segments of stored programs are stored in a database including timing information associated with segments of each respective stored program; and wherein
a match may be found when timing information of said segments produced in said grouping step matches timing information of segments stored in said database.
18. The method of claim 1 , wherein match scores are determined based on Euclidean distances between frequency coefficient segments of said media program and stored programs.
19. The method of claim 17 , further comprising identifying said media program according to a media program identifier stored in said database associated with a best matching score when matching segments are found in said database.
20. The method of claim 19 , further comprising determining a speed differential between said media program and an identified stored media program.
21. The method of claim 1 , wherein said matching score for a program Pi is determined by
P
i
=
1
z
∑
j
=
1
z
f
(
S
j
=
1
′
-
S
j
(
P
1
)
)
.
22. The method of claim 1 , further comprising:
repeating said filtering, grouping, selecting, comparing and identifying; and
determining, in the event of another match, whether said identified program is the same program determined prior to said repetition or a different program.
23. The method of claim 22 , wherein said determining step is based on an overlap score.
24. The method of claim 23 , wherein overlap score is calculated between said program determined prior to said repetition, P 0 , and said program determined during said repetition, P 1 , is calculated as
Overlap score=( t end− t begin)/(end time of P 1−beginning time of P 1)
wheretend is min(end time of P 0 , P 1 ); and
tbegin is max(beginning time of P 0 , P 1 ).
25. A tangible and non-transient computer readable storage medium storing instructions which, when executed by a computer, adapt the operation of the computer to provide a method for use in recognizing the content of a media program, the method comprising:
filtering each first frequency domain representation of blocks of said media program using a plurality of filters to develop a respective second frequency domain representation of each of said blocks of said media program, said second frequency domain representation of each of said blocks having a reduced number of frequency coefficients with respect to said first frequency domain representation;
grouping frequency coefficients of said second frequency domain representation of said blocks to form frequency coefficient segments;
selecting a plurality of said segments as representing said media program;
comparing said selected segments to frequency coefficient segments of stored programs to provide corresponding matching scores; and
determining said media program using said matching scores,
wherein said first frequency domain representation of blocks of said media program is developed by:
digitizing an audio representation of said media program;
dividing the digitized audio representation into time domain blocks of a prescribed number of samples;
smoothing said time domain blocks using a filter; and
converting said smoothed time domain blocks into frequency domain blocks, wherein said smoothed time domain blocks are represented by frequency coefficients.