IP Library Granted Patent US 11,051,055
Granted Patent B1
US 11,051,055 · App. 16/868,651 · Granted Jun 29, 2021

Automatic content recognition (ACR) fingerprinting and video encoding

Inventors: Andrew Ashbacher (Santa Cruz, CA); Marcus Liassides (Cardiff, GB)
Assignee: Roku, Inc.
H04N21/23418H04N21/2225H04N21/2343H04N21/2353H04N21/4586H04N21/84H04N21/23602
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,051,055
App. No.
16/868,651
Granted
Jun 29, 2021
Kind
B1
Abstract

A processing device and method are disclosed to generate automatic content recognition (ACR) fingerprints. A device can include a processing device to execute a capture daemon. The capture daemon may include a capturer to receive media content from a content feed and to generate content frames of the media content. The capture daemon can also include ACR fingerprinter to generate ACR fingerprint information for the content frames. The ACR fingerprinter may communicate the ACR fingerprint information to an ACR server via a first communication channel. The processing device may determine when the ACR fingerprinter fails; cease communicating the ACR fingerprint information to the ACR server; and restart the ACR fingerprinter.

Claims (46)

1. A device comprising:

a capture interface; and

a processing device coupled to the capture interface, the processing device configured to execute a capture daemon, the capture daemon comprising:

a capturer configured to: (i) receive media content from a content feed, (ii) receive a status message from a broadcast server indicating a status of the broadcast server to receive streaming content, (iii) generate content frames of the media content, and (iv) send the generated content frames for encoding and fingerprinting; and

an automatic content recognition (ACR) fingerprinter configured to: (i) receive one or more of the generated content frames from the capturer, (ii) generate ACR fingerprint information for the received content frames, (iii) communicate the ACR fingerprint information to an ACR server via a first communication channel, while determining whether the ACR fingerprinter fails, and (iv) when the ACR fingerprinter fails and while the content frames are encoded: (a) cease the communication of the ACR fingerprint information from the ACR fingerprinter to the ACR server, and (b) restart the ACR fingerprinter.

2. The device of claim 1 , wherein the capture interface is configured to convert the media content to a format that is compatible with the capturer.

3. The device of claim 1 , wherein the capture interface includes a digital interface port to receive the media content.

4. The device of claim 3 , wherein the capturer is configured to:

receive media content from the content feed via the digital interface port; and

communicate capture criterion to the capture interface, wherein the capture criterion is a criterion to obtain media content from the content feed.

5. The device of claim 1 , wherein the processing device is further configured to execute a software updater to update the software executed by the processing device.

6. The device of claim 5 , wherein the software updater is configured to:

communicate with the broadcast server to determine a time when an update is scheduled; and

update the software at the time.

7. The device of claim 1 , wherein the capturer is configured to communicate with the broadcast server to indicate when the media content is being received.

8. The device of claim 1 , wherein the capture daemon further comprises an encoder to encode the media content in parallel with the ACR fingerprinter generating the ACR fingerprint information.

9. The device of claim 1 , wherein the ACR fingerprinter is configured to:

determine the ACR fingerprint information for a first media consumption device using a first ACR algorithm; and

determine the ACR fingerprint information for a second media consumption device using a second ACR algorithm.

10. The device of claim 1 , wherein the capturer is configured to send a system metric to the broadcast server, wherein the system metric comprises at least one of central processing unit (CPU) utilization, memory utilization, drive utilization, or operating temperature.

11. The device of claim 1 , wherein sending the generated content frames for encoding and fingerprinting comprises sending a first set of the generated content frames for encoding while sending, in parallel, a second set of the generated content frames for fingerprinting.

12. A device comprising:

a processing device configured to execute a capture daemon and an encoder, the encoder configured to receive content frames from a capturer and encode media content based on encoding criteria, the capture daemon comprising:

the capturer, wherein the capturer is configured to: (i) receive media content from a content feed, (ii) generate content frames of the media content, and (iii) send the generated content frames to the encoder for encoding and to a fingerprinter for fingerprinting; and

an automatic content recognition (ACR) fingerprinter, wherein the ACR fingerprinter is configured to: (i) generate ACR fingerprint information for the content frames received from the capturer, (ii) communicate the ACR fingerprint information to an ACR server, while determining whether the ACR fingerprinter fails, and (iii) when the ACR fingerprinter fails and while the encoder continues encoding content frames: (a) cease the communication of the ACR fingerprint information from the ACR fingerprinter to the ACR server, and (b) restart the ACR fingerprinter.

13. The device of claim 12 , wherein the processing device further comprises a capture interface configured to convert the media content to a format that is compatible with the capturer.

14. The device of claim 13 , wherein:

the capturer is configured to communicate a capture criterion to the capture interface; and

the capture criterion is criterion to obtain media content from the content feed.

15. The device of claim 12 , wherein the processing device is further configured to execute the encoder to adjust for a display size of a media consumption device, a processing capability of the media consumption device, a network condition of the media consumption device, or a battery capability of the media consumption device.

16. A method comprising:

receiving media content at a processing device;

receiving, by the processing device, a status message from a broadcast server indicating a status of the broadcast server to receive streaming content;

generating, by the processing device, content frames from the received media content;

executing, by the processing device, an automatic content recognition (ACR) fingerprinter;

generating, by the processing device using the ACR fingerprinter, ACR fingerprinting information based on a first set of the generated content frames;

communicating, by the processing device, the ACR fingerprint information to an ACR cloud system, while determining whether the ACR fingerprinter fails; and

when the ACR fingerprinter fails and while a second set of the generated content frames are encoded: (i) ceasing communicating the ACR fingerprint information to the ACR cloud system, and (ii) restarting the ACR fingerprinter.

17. The method of claim 16 , further comprising:

encoding, by the processing device, one or more of the generated content frames based on encoding criteria to obtain encoded media content; and

communicating the encoded media content from the processing device to the broadcast server while the ACR fingerprinter restarts.

18. The method of claim 16 , further comprising encoding, by the processing device, one or more of the generated content frames in parallel with generating the ACR fingerprint information.

19. The method of claim 16 , further comprising:

determining, by the processing device, the ACR fingerprint information for a first media consumption device using a first ACR algorithm; and

determining, by the processing device, the ACR fingerprint information for a second media consumption device using a second ACR algorithm.

20. The method of claim 16 , wherein generating the content frames from the received media content comprises generating the content frames at a first resolution for fingerprinting and at a second resolution for encoding.

Assignments (11)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: SORENSON MEDIA, INC.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 054170/0552 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: ASHBACHER, ANDREW; LIASSIDES, MARCUS
To: SORENSON MEDIA, INC.
Reel/Frame 054170/0536 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
Continuity (3)
Continuation 15457483 · Mar 13, 2017
Continuation 14839339 · Aug 28, 2015
Provisional Application 62095539 · Dec 22, 2014