Extracting features of audio signal content to provide reliable identification of the signals
Signatures that can be used to identify video and audio content are generated from the content by generating measures of dissimilarity between features of corresponding groups of pixels in frames of video content and by generating low-resolution time-frequency representations of audio segments. The signatures are generated by applying a hash function to intermediate values derived from the measures of dissimilarity and to the low-resolution time-frequency representations. The generated signatures may be used in a variety of applications such as restoring synchronization between video and audio content streams and identifying copies of original video and audio content. The generated signatures can provide reliable identifications despite intentional and unintentional modifications to the content.
1. A method for generating a signature that identifies content of an audio signal, wherein the method performed by a device comprises:
obtaining, by a device, a time-frequency representation of a set of blocks within a sequence of blocks of the audio signal, wherein the time-frequency representation comprises sets of spectral values, each set of spectral values representing all spectral components within at least a portion of the bandwidth of the audio signal in a respective block in the set of blocks;
deriving, by a device, intermediate values from intensities of all the spectral values arranged in groups of one or more spectral values within a respective set of spectral values; and
generating, by a device the signature that identifies content of the audio signal by projecting the intermediate values onto a set of random vectors, wherein the signature is represented by bits and each bit of the signature is derived from contributions from all of the intermediate values;
wherein each respective vector in the set of random vectors has vector elements with values that are obtained from a difference between uniformly distributed random variables within a range from zero to one and an average of the uniformly distributed random variables for all vector elements of the respective vector;
the projection of the intermediate values onto a respective random vector is obtained from an inner product of the intermediate values with the vector elements of the respective vector; and
each component of the signature has either a first value when the projection of the intermediate values onto a corresponding random vector is greater than a threshold or has a second value when the projection of the intermediate values onto the corresponding random vector is less than a threshold, wherein the threshold is equal to a median of the projections of intermediate values onto the set of random vectors.
2. The method of claim 1 , wherein:
the time-frequency representation is obtained by applying a time-to-frequency transform to each block of the audio signal in the set of blocks to obtain a respective set of spectral values; and
a respective intermediate value is derived by calculating an average intensity of the one or more spectral values in a group within the respective set of spectral values.
3. The method of claim 1 , wherein the groups of spectral values have numbers of spectral values that vary with frequency.
4. The method of claim 3 , wherein the groups of spectral values for higher frequencies have a greater number of spectral values.
5. The method of claim 1 , wherein each component of the signature is derived from the projection of the intermediate values onto a respective random vector.
6. An apparatus for generating a signature that identifies content of an audio signal, wherein the apparatus comprises:
means for obtaining a time-frequency representation of a set of blocks within a sequence of blocks of the audio signal, wherein the time-frequency representation comprises sets of spectral values, each set of spectral values representing all spectral components within at least a portion of the bandwidth of the audio signal in a respective block in the set of blocks;
means for deriving intermediate values from intensities of all the spectral values arranged in groups of one or more spectral values within a respective set of spectral values; and
means for generating the signature that identifies content of the audio signal by projecting the intermediate values onto a set of random vectors, wherein the signature is represented by bits and each bit of the signature is derived from contributions from all of the intermediate values;
wherein each respective vector in the set of random vectors has vector elements with values that are obtained from a difference between uniformly distributed random variables within a range from zero to one and an average of the uniformly distributed random variables for all vector elements of the respective vector;
the projection of the intermediate values onto a respective random vector is obtained from an inner product of the intermediate values with the vector elements of the respective vector; and
each component of the signature has either a first value when the projection of the intermediate values onto a corresponding random vector is greater than a threshold or has a second value when the projection of the intermediate values onto the corresponding random vector is less than a threshold, wherein the threshold is equal to a median of the projections of intermediate values onto the set of random vectors.
7. The apparatus of claim 6 , wherein:
the time-frequency representation is obtained by applying a time-to-frequency transform to each block of the audio signal in the set of blocks to obtain a respective set of spectral values; and
a respective intermediate value is derived by calculating an average intensity of the one or more spectral values in a group within the respective set of spectral values.
8. The apparatus of claim 6 , wherein the groups of spectral values have numbers of spectral values that vary with frequency.
9. The apparatus of claim 8 , wherein the groups of spectral values for higher frequencies have a greater number of spectral values.
10. The apparatus of claim 6 , wherein each component of the signature is derived from the projection of the intermediate values onto a respective random vector.
11. A non-transitory computer readable storage medium that records a program of instructions that is executable by a device to perform a method for generating a signature that identifies content of an audio signal, wherein the method comprises:
obtaining a time-frequency representation of a set of blocks within a sequence of blocks of the audio signal, wherein the time-frequency representation comprises sets of spectral values, each set of spectral values representing all spectral components within at least a portion of the bandwidth of the audio signal in a respective block in the set of blocks;
deriving intermediate values from intensities of all the spectral values arranged in groups of one or more spectral values within a respective set of spectral values; and
generating the signature that identifies content of the audio signal by projecting the intermediate values onto a set of random vectors, wherein the signature is represented by bits and each bit of the signature is derived from contributions from all of the intermediate values;
wherein each respective vector in the set of random vectors has vector elements with values that are obtained from a difference between uniformly distributed random variables within a range from zero to one and an average of the uniformly distributed random variables for all vector elements of the respective vector;
the projection of the intermediate values onto a respective random vector is obtained from an inner product of the intermediate values with the vector elements of the respective vector; and
each component of the signature has either a first value when the projection of the intermediate values onto a corresponding random vector is greater than a threshold or has a second value when the projection of the intermediate values onto the corresponding random vector is less than a threshold, wherein the threshold is equal to a median of the projections of intermediate values onto the set of random vectors.
12. The storage non-transitory computer readable storage medium of claim 11 , wherein:
the time-frequency representation is obtained by applying a time-to-frequency transform to each block of the audio signal in the set of blocks to obtain a respective set of spectral values; and
a respective intermediate value is derived by calculating an average intensity of the one or more spectral values in a group within the respective set of spectral values.
13. The non-transitory computer readable storage medium of claim 11 , wherein the groups of spectral values have numbers of spectral values that vary with frequency.
14. The non-transitory computer readable medium of claim 13 , wherein the groups of spectral values for higher frequencies have a greater number of spectral values.
15. The non-transitory computer readable storage medium of claim 11 , wherein each component of the signature is derived from the projection of the intermediate values onto a respective random vector.