Video fingerprinting based on fourier transform of histogram
View Patent ↗A content device and method is disclosed to include a processing device to process streaming video content. A fingerprinter receives captured frames of the streaming video content and, for each frame of a plurality of the captured frames, generates a one-dimensional histogram function of pixel values and transforms the histogram function with a Fast Fourier Transform (FFT), to generate a plurality of complex values for the frame. The fingerprinter further, for each of the plurality of complex values, assigns a binary one (“1”) when a real part of the complex value is greater than zero (“0”) and assigns a binary zero (“0”) when the real part is less than or equal to zero, to generate a plurality of bits. The fingerprinter further concatenates a specific number of the bits to generate a fingerprint for the frame.
1. A content device comprising:
a non-transitory memory storing instructions;
a capture interface including a port configured to receive streaming video content from a content provider; and
a processing device operatively coupled to the non-transitory memory and the capture interface and configured to execute the instructions, the executed instructions causing the processing device to perform operations comprising:
splitting the streaming video content into a first thread and a second thread;
encoding the first thread of the streaming video content into encoded video content;
sending the encoded video content from the processing device to a broadcast server, the broadcast server in response to receiving the encoded video content, delivering the encoded video content to a media consumption device;
capturing frames of the second thread of the streaming video content; and
for each captured frame of the second thread of the streaming video content, fingerprinting the corresponding captured frame simultaneously or in parallel with the encoding of the first thread of the streaming video content by:
generating a one-dimensional histogram function of pixel values;
transforming the histogram function with a Fast Fourier Transform (FFT) to generate a plurality of complex values for the corresponding captured frame;
for each of the plurality of complex values, generating a plurality of bits by assigning a binary one (“1”) when a real part of the complex value is greater than zero (“0”) and assigning a binary zero (“0”) when the real part is less than or equal to zero; and
concatenating a specific number of the plurality of bits to generate a fingerprint for the corresponding captured frame; and
sending the fingerprint for the corresponding captured frame to an automatic content recognition (ACR) server, the ACR server in response to receiving the fingerprint, storing the fingerprint in a fingerprint database.
2. The content device of claim 1 , wherein the histogram function includes an x axis representing specific pixel values and a y axis representing a number of pixels at each respective specific pixel value.
3. The content device of claim 2 , wherein each specific pixel value of the histogram function comprises a corresponding Y value of a YUV color space.
4. The content device of claim 1 , wherein the operations further comprise, when fingerprinting the corresponding captured frame, indexing the plurality of complex values from zero (“0”) to 255.
5. The content device of claim 1 , wherein the generated fingerprint is 128 bits long or 256 bits long.
6. The content device of claim 1 , wherein the ACR server, in response to receiving the fingerprint, is further configured to store the fingerprint in relation to corresponding broadcast schedule metadata.
7. A method comprising:
receiving, at a processing device of a content device, streaming video content from a content provider via a capturer interface, the capturer interface operatively coupled to the processing device;
splitting, by the processing device, the streaming video content into a first thread and a second thread;
encoding, by the processing device, the first thread of the streaming video content into encoded video content;
sending the encoded video content from the processing device to a broadcast server, the broadcast server in response to receiving the encoded video content, delivering the encoded video content to a media consumption device;
capturing, by the processing device, frames of the second thread of the streaming video content; and
for each captured frame of the second thread of the streaming video content, fingerprinting, by the processing device, the corresponding captured frame simultaneously or in parallel with the encoding of the first thread of the streaming video content by:
generating a one-dimensional histogram function of pixel values;
transforming the histogram function with a Fast Fourier Transform (FFT) to generate a plurality of complex values for the corresponding captured frame;
for each of the plurality of complex values, generating a plurality of bits by assigning a binary one (“1”) when a real part of the complex value is greater than zero (“0”) and assigning a binary zero (“0”) when the real part is less than or equal to zero;
concatenating a specific number of the plurality of bits to generate a fingerprint for the corresponding captured frame; and
sending the fingerprint for the corresponding captured frame to an automatic content recognition (ACR) server, the ACR server in response to receiving the fingerprint, storing the fingerprint in a fingerprint database.
8. The method of claim 7 , wherein the histogram function includes an x axis representing specific pixel values and a y axis representing a number of pixels at each respective specific pixel value.
9. The method of claim 8 , wherein each specific value of the histogram function comprises a corresponding Y value of a YUV color space.
10. The method of claim 7 , further comprising, when fingerprinting the corresponding captured frame, indexing, by the processor device, the plurality of complex values from zero (“0”) to 255.
11. The method of claim 7 , wherein the generated fingerprint is 128 bits long or 256 bits long.
12. The method of claim 7 , ACR server in response to receiving the fingerprint, is further configured to store the fingerprint in relation to corresponding broadcast schedule metadata.
13. A non-transitory machine-readable storage medium storing instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving streaming video content from a content provider via a capturer interface, the capturer interface operatively coupled to the processing device;
splitting the streaming video content into a first thread and a second thread;
encoding the first thread of the streaming video content into encoded video content;
sending the encoded video content from the processing device to a broadcast server, the broadcast server in response to receiving the encoded video content, delivering the encoded video content to a media consumption device;
capturing frames of the second thread of the streaming video content; and
for each captured frame of the second thread of the streaming video content, fingerprinting the corresponding captured frame simultaneously or in parallel with the encoding of the first thread of the streaming video content by:
generating a one-dimensional histogram function of pixel values;
transforming the histogram function with a Fast Fourier Transform (FFT)) to generate a plurality of complex values for the corresponding captured frame;
for each of the plurality of complex values, generating a plurality of bits by assigning a binary one (“1”) when a real part of the complex value is greater than zero (“0”) and assigning a binary zero (“0”) when the real part is less than or equal to zero;
concatenating a specific number of the plurality of bits to generate a fingerprint for the corresponding captured frame; and
sending the fingerprint for the corresponding captured frame to an automatic content recognition (ACR) server, the ACR server in response to receiving the fingerprint, storing the fingerprint in a fingerprint database.
14. The non-transitory machine-readable storage medium of claim 13 , wherein the histogram function includes an x axis representing specific pixel values and a y axis representing a number of pixels at each respective specific pixel value.
15. The non-transitory machine-readable storage medium of claim 14 ,
wherein each specific pixel value of the histogram function comprises a corresponding Y value of a YUV color space.
16. The non-transitory machine-readable storage medium of claim 13 , wherein the operations further comprise, when fingerprinting the corresponding captured frame, indexing the plurality of complex values from zero (“0”) to 255.
17. The non-transitory machine-readable storage medium of claim 13 , wherein the generated fingerprint is 128 bits long or 256 bits long.