IP Library Granted Patent US 11,172,262
Granted Patent B2
US 11,172,262 · App. 16/495,337 · Granted Nov 9, 2021

Enabling interactive control of live television broadcast streams

Inventor: Matthew Grover (Draper, UT)
Assignee: Roku, Inc.
H04N21/47217H04N21/2187H04N21/222H04N21/23418H04N21/4312H04N21/4355H04N21/44008H04N21/458H04N21/4532H04N21/4622H04N21/6587H04N21/812H04N21/8358H04N21/84H04N21/8545H04N21/8586
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,172,262
App. No.
16/495,337
Granted
Nov 9, 2021
Kind
B2
Abstract

The disclosure provides a method for interactive control of live television broadcast streams. The method includes receiving a streaming live media feed ( 103 ) from a television broadcaster ( 102 ) and displaying an overlay ( 212 ) on media content (M) of the streaming live media feed ( 103 ). The overlay ( 212 ) includes a play control option unassociated with an intermediary feed source of the live media feed between the television broadcaster and the media device ( 115 ). The method further includes receiving a first selection input of the play control option and determining that the selection input corresponds to a play mode at a first location within the media content (M) of the streaming live media feed by determining that a first fingerprint ( 128, 208, 306 ) fails to match a second fingerprint ( 128, 208, 306 ). Furthermore, the method includes overlaying targeted overlay content on the streaming live media feed.

Claims (46)

1. A method comprising:

receiving, at data processing hardware of a media device, a streaming live media feed from a television broadcaster, the streaming live media feed comprising media content;

displaying, by the data processing hardware of the media device, an overlay on the media content of the streaming live media feed, the overlay containing control options for controlling playing of the streaming live media feed, including pause, play, fast-forward, and rewind, and the overlay being unassociated with an intermediary feed source of the streaming live media feed between the television broadcaster and the media device;

detecting, by the data processing hardware of the media device, a first selection of a first one of the control options of the overlay, the first selection being to fast-forward or rewind the streaming live media feed, and executing, by the data processing hardware of the media device, the first selected control option;

subsequently detecting, by the data processing hardware of the media device, a second selection of a second one of the control options of the overlay, the second selection being to stop the fast-forwarding or rewinding of the streaming live media feed and to resume playing of the streaming live media feed, and executing, by the data processing hardware of the media device, the second selected control option;

using, by the data processing hardware of the media device, fingerprint-based automatic content recognition to identify content of the streaming live media feed at a location in the streaming live media feed where the playing resumed, wherein using fingerprint-based automatic content recognition to identify content of the streaming live media feed at the location in the streaming live media feed where the playing resumed comprises (a) comparing query fingerprint data representing sequential frames of content of the streaming live media feed at the media device with frame fingerprint data representing the streaming live media feed provided by the television broadcaster and (b) detecting, based on the comparing, at least one fingerprint match; and

overlaying, by the data processing hardware of the media device, targeted overlay content on the streaming live media feed, the targeted overlay content being selected based on the identified content of the streaming live media feed where playing resumed.

2. The method of claim 1 , further comprising:

sending, by the media device, an overlay request to an overlay decision engine, requesting the targeted overlay content.

3. The method of claim 2 , wherein the overlay request includes a media content identifier.

4. The method of claim 1 , wherein the targeted overlay content is contextually relevant to the identified content of the streaming live media feed at a location in the streaming live media feed where the playing resumed.

5. The method of claim 4 , further comprising:

determining, based on a user profile, the targeted overlay content contextually relevant to the identified content at the location in the streaming live media feed where the playing resumed.

6. The method of claim 1 , wherein the intermediary feed source of the streaming live media feed is selected from the group consisting of aerial, satellite, internet-enabled, and set top box.

7. The method of claim 1 , further comprising buffering, by the data processing hardware of the media device, the content of the streaming live media feed for at least a period of time, the period of time having a duration between the first selection and the second selection input.

8. The method of claim 1 , further comprising:

receiving, at the data processing hardware of the media device, the frame fingerprint data representing the streaming live media feed.

9. The method of claim 1 , wherein the overlay comprises a hypertext markup language browser (HTML) overlay.

10. A media device comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the media device to perform operations comprising:

receiving a streaming live media feed from a television broadcaster, the streaming live media feed comprising media content,

displaying an overlay on the media content of the streaming live media feed, the overlay containing control options for controlling playing of the streaming live media feed, including pause, play, fast-forward, and rewind, and the overlay being unassociated with an intermediary feed source of the streaming live media feed between the television broadcaster and the media device,

detecting a first selection of a first one of the control options of the overlay, the first selection being to fast-forward or rewind the streaming live media feed, and executing, by the data processing hardware of the media device, the first selected control option,

subsequently detecting a second selection of a second one of the control options of the overlay, the second selection being to stop the fast-forwarding or rewinding of the streaming live media feed and to resume playing of the streaming live media feed, and executing the second selected control option,

using fingerprint-based automatic content recognition to identify content of the streaming live media feed at a location in the streaming live media feed where the playing resumed, wherein using fingerprint-based automatic content recognition to identify content of the streaming live media feed at the location in the streaming live media feed where the playing resumed comprises (a) comparing query fingerprint data representing sequential frames of content of the streaming live media feed at the media device with frame fingerprint data representing the streaming live media feed provided by the television broadcaster and (b) detecting, based on the comparing, at least one fingerprint match, and

overlaying targeted overlay content on the streaming live media feed, the targeted overlay content being selected based on the identified content of the streaming live media feed where playing resumed.

11. The media device of claim 10 , wherein the operations further comprise:

sending an overlay request to an overlay decision engine, requesting the targeted overlay content.

12. The media device of claim 11 , wherein the overlay request includes a media content identifier.

13. The media device of claim 10 , wherein the targeted overlay content is contextually relevant to the identified content of the streaming live media feed at a location in the streaming live media feed where the playing resumed.

14. The media device of claim 13 , wherein the operations further comprise:

determining, based on a user profile, the targeted overlay content contextually relevant to the identified content at the location in the streaming live media feed where the playing resumed.

15. The media device of claim 10 , wherein the intermediary feed source of the streaming live media feed is selected from the group consisting of aerial, satellite, internet-enabled, and set top box.

16. The media device of claim 10 , wherein the operations further comprise buffering the content of the streaming live media feed for at least a period of time, the period of time having a duration between the first selection and the second selection input.

17. The media device of claim 10 , wherein the operations further comprise:

receiving the frame fingerprint data representing the streaming live media feed.

18. The media device of claim 10 , wherein the overlay comprises a hypertext markup language browser (HTML) overlay.

19. A non-transitory computer-readable storage medium storing instructions executable to cause data processing hardware of a media device to carry out operations comprising:

receiving a streaming live media feed from a television broadcaster, the streaming live media feed comprising media content;

displaying an overlay on the media content of the streaming live media feed, the overlay containing control options for controlling playing of the streaming live media feed, including pause, play, fast-forward, and rewind, and the overlay being unassociated with an intermediary feed source of the streaming live media feed between the television broadcaster and the media device;

detecting a first selection of a first one of the control options of the overlay, the first selection being to fast-forward or rewind the streaming live media feed, and executing, by the data processing hardware of the media device, the first selected control option;

subsequently detecting a second selection of a second one of the control options of the overlay, the second selection being to stop the fast-forwarding or rewinding of the streaming live media feed and to resume playing of the streaming live media feed, and executing the second selected control option;

using fingerprint-based automatic content recognition to identify content of the streaming live media feed at a location in the streaming live media feed where the playing resumed, wherein using fingerprint-based automatic content recognition to identify content of the streaming live media feed at the location in the streaming live media feed where the playing resumed comprises (a) comparing query fingerprint data representing sequential frames of content of the streaming live media feed at the media device with frame fingerprint data representing the streaming live media feed provided by the television broadcaster and (b) detecting, based on the comparing, at least one fingerprint match; and

overlaying targeted overlay content on the streaming live media feed, the targeted overlay content being selected based on the identified content of the streaming live media feed where playing resumed.

20. The non-transitory computer-readable medium of claim 19 , wherein the targeted overlay content is contextually relevant to the identified content of the streaming live media feed at a location in the streaming live media feed where the playing resumed.

Assignments (11)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2020
From: GROVER, MATTHEW
To: SORENSON MEDIA, INC.
Reel/Frame 054130/0871 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2020
From: SORENSON MEDIA, INC.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 054131/0400 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →