IP Library Granted Patent US 11,546,656
Granted Patent B2
US 11,546,656 · App. 16/502,649 · Granted Jan 3, 2023

Video fingerprinting based on Fourier transform of histogram

Inventor: Juikun Chen (West Jordan, UT)
Assignee: ROKU, INC.
H04N21/440245G06F16/00G06F16/7847G06V10/431G06V10/50G06V20/46G06V20/48H04N21/222H04N21/23424H04N21/2407H04N21/26258H04N21/435H04N21/437H04N21/44008H04N21/812H04N21/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,546,656
App. No.
16/502,649
Granted
Jan 3, 2023
Kind
B2
Abstract

A content device and method is disclosed to include a processing device to process streaming video content. A fingerprinter receives captured frames of the streaming video content and, for each frame of a plurality of the captured frames, generates a one-dimensional histogram function of pixel values and transforms the histogram function with a Fast Fourier Transform (FFT), to generate a plurality of complex values for the frame. The fingerprinter further, for each of the plurality of complex values, assigns a binary one (“1”) when a real part of the complex value is greater than zero (“0”) and assigns a binary zero (“0”) when the real part is less than or equal to zero, to generate a plurality of bits. The fingerprinter further concatenates a specific number of the bits to generate a fingerprint for the frame.

Claims (30)

1. A content device comprising:

a non-transitory memory storing instructions; and

a processing device in communication with the non-transitory memory and configured to execute the instructions, the instructions when executed causing the processing device to perform operations comprising:

receiving a video frame;

generating a histogram function that indicates a quantitative distribution of pixel values within the video frame;

transforming the histogram function to generate complex values that correspond respectively to the pixel values; and

for each of the complex values, generating a bit that indicates whether a real part of the complex value is greater than zero,

wherein generating the bit for each of the complex values comprises generating a fingerprint, wherein the operations further comprise communicating the fingerprint to a server.

2. The content device of claim 1 , wherein transforming the histogram function comprises transforming the histogram function using a Fast Fourier Transform (FFT).

3. The content device of claim 1 , wherein the pixel values correspond to Y values of a YUV color space.

4. The content device of claim 1 , wherein the operations further comprise indexing the complex values from 0 to 255.

5. The content device of claim 1 , wherein the fingerprint includes 128 bits or 256 bits.

6. The content device of claim 1 , wherein the video frame comprises encoded video content.

7. The content device of claim 1 , wherein the video frame comprises video within a YUV 4:2:2 color space.

8. The content device of claim 1 , wherein the operations further comprise converting the video frame by scaling a resolution of the video frame to a threshold resolution level.

9. The content device of claim 8 , wherein the converting further comprises converting the video frame to a YUV 4:2:0 color space.

10. A method comprising:

receiving, at a processing device of a content device, a video frame;

generating a histogram function that indicates a quantitative distribution of pixel values within the video frame;

transforming the histogram function to generate complex values that correspond respectively to the pixel values; and

for each of the complex values, generating a bit that indicates whether a real part of the complex value is greater than zero,

wherein generating the bit for each of the complex values comprises generating a fingerprint, the method further comprising communicating the fingerprint to a server.

11. The method of claim 10 , wherein transforming the histogram function comprises transforming the histogram function using a Fast Fourier Transform (FFT).

12. The method of claim 10 , wherein the pixel values correspond to Y values of a YUV color space.

13. The method of claim 10 , further comprising indexing the complex values from 0 to 255.

14. The method of claim 10 , wherein the fingerprint includes 128 bits or 256 bits.

15. The method of claim 10 , wherein the video frame comprises encoded video content.

16. The method of claim 10 , wherein the video frame comprises video within a YUV 4:2:2 color space.

17. The method of claim 10 , further comprising converting the video frame by scaling a resolution of the video frame to a threshold resolution level.

18. The method of claim 17 , wherein the converting further comprises converting the video frame to a YUV 4:2:0 color space.

Assignments (11)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 3, 2019
From: CHEN, JUIKUN
To: SORENSON MEDIA, INC.
Reel/Frame 049665/0328 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 3, 2019
From: SORENSON MEDIA, INC.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 049665/0378 →