Generating a sequence of audio fingerprints at a set top box
View Patent ↗A computer-implemented method for generating audio fingerprints at a set top box that is communicatively coupled to a television and reporting the audio fingerprints from the set top box to a remote server is disclosed. After receiving a video signal and an audio signal associated with the video signal from a TV content provider, both of which correspond to a TV program being played on the television, the set top box generates a plurality of audio fingerprints from the audio signal, at least one of which corresponds to a predefined video scene transition detected in the video signal. The set top box then sends the plurality of audio fingerprints to the remote server for determining TV program information associated with the TV program.
1. A method, comprising:
at a first electronic device including one or more processors and memory:
receiving, from a TV content provider, a first video signal and a first audio signal associated with the first video signal, wherein the first video signal and the first audio signal both correspond to a same TV program;
generating a first sequence of audio fingerprints from the first audio signal, wherein each audio fingerprint in the first sequence has a corresponding video scene transition type detected in the first video signal, and wherein each video scene transition type indicates change in luminosity;
sending the first sequence of audio fingerprints and the corresponding video scene transition types to a remote server for determining TV program information associated with the TV program, wherein the first sequence of audio fingerprints and the corresponding video scene transition types enable the remote server to match the first sequence of audio fingerprints to a second sequence of audio fingerprints using video scene transition types corresponding to each of the audio fingerprints in the second sequence, and wherein the second sequence of audio fingerprints and corresponding video scene transition types are associated with a known TV program; and
receiving from the remote server information about the TV program based on the matching.
2. The method of claim 1 , wherein generating the first sequence of audio fingerprints from the audio signal further includes:
periodically capturing a segment of audio signal from the audio signal; and
generating an audio fingerprint from the captured audio signal segment.
3. The method of claim 1 , wherein each audio fingerprint in the first and second sequences is a vector including one or more of average zero crossing rate, estimated tempo, average spectrum, spectral flatness, prominent tones across a set of bands, and bandwidth.
4. A first electronic device comprising:
one or more processors;
memory; and
a plurality of program modules stored in the memory and configured for execution by the one or more processors, wherein the plurality of program modules include instructions for:
receiving, from a TV content provider, a first video signal and a first audio signal associated with the first video signal, wherein the first video signal and the first audio signal both correspond to a same TV program;
generating a first sequence of audio fingerprints from the first audio signal, wherein each audio fingerprint in the first sequence has a corresponding video scene transition type detected in the first video signal, and wherein each video scene transition type indicates change in luminosity;
sending the first sequence of audio fingerprints and the corresponding video scene transition types to a remote server for determining TV program information associated with the TV program, wherein the first sequence of audio fingerprints and the corresponding video scene transition types enable the remote server to match the first sequence of audio fingerprints to a second sequence of audio fingerprints using video scene transition types corresponding to each of the audio fingerprints in the second sequence and wherein the second sequence of audio fingerprints and corresponding video scene transition types are associated with a known TV program; and
receiving from the remote server information about the TV program based on the matching.
5. The first electronic device of claim 4 , wherein the instructions for generating the first sequence of audio fingerprints from the audio signal further comprise instructions for:
periodically capturing a segment of audio signal from the audio signal; and
generating an audio fingerprint from the captured audio signal segment.
6. The first electronic device of claim 4 , wherein each audio fingerprint in the first and second sequences is a vector including one or more of average zero crossing rate, estimated tempo, average spectrum, spectral flatness, prominent tones across a set of bands, or bandwidth.
7. A non-transitory computer readable storage medium storing one or more program modules configured for execution by a first electronic device that includes one or more processors and memory, the one or more programs comprising instructions for:
receiving, from a TV content provider, a first video signal and a first audio signal associated with the first video signal, wherein the first video signal and the first audio signal both correspond to a same TV program;
generating a first sequence of audio fingerprints from the first audio signal, wherein each audio fingerprint in the first sequence has a corresponding video scene transition type detected in the first video signal, and wherein each video scene transition type indicates change in luminosity;
sending the first sequence of audio fingerprints and the corresponding video scene transition types to a remote server for determining TV program information associated with the TV program, wherein the first sequence of audio fingerprints and the corresponding video scene transition types enable the remote server to match the first sequence of audio fingerprints to a second sequence of audio fingerprints using video scene transition types corresponding to each of the audio fingerprints in the second sequence, and wherein the second sequence of audio fingerprints and corresponding video scene transition types are associated with a known TV program; and
receiving from the remote server information about the TV program based on the matching.
8. The non-transitory computer readable storage medium of claim 7 , wherein the instructions for generating the first sequence of audio fingerprints from the audio signal further comprise instructions for:
periodically capturing a segment of audio signal from the audio signal; and
generating an audio fingerprint from the captured audio signal segment.
9. The non-transitory computer readable storage medium of claim 7 , wherein each audio fingerprint in the first and second sequences is a vector including one or more of average zero crossing rate, estimated tempo, average spectrum, spectral flatness, prominent tones across a set of bands, and bandwidth.