IP Library Granted Patent US 11,109,074
Granted Patent B2
US 11,109,074 · App. 16/815,720 · Granted Aug 31, 2021

Method and device for generating and detecting a fingerprint functioning as a trigger marker in a multimedia signal

Inventors: Job Cornelis Oostveen (Haren, NL); Warner Rudolph Theophile Ten Kate (Eindhoven, NL); Adrianus Johannes Maria Denissen (Eindhoven, NL); David K. Roberts (Cheddar, GB)
Assignee: Roku, Inc.
H04N21/23418G06F21/6227H04N19/467H04N21/23109H04N21/433H04N21/435H04N21/4394H04N21/44008H04N21/4586H04N21/6587H04N21/8126H04N21/8133
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,109,074
App. No.
16/815,720
Granted
Aug 31, 2021
Kind
B2
Abstract

A device is configured to perform a method that detects a trigger marker for an action corresponding to a segment of a multimedia signal. A fingerprint is generated based on the segment of the multimedia signal at a trigger time point. The generated fingerprint is stored in a database and communicated to the device. During playback of the multimedia signal, fingerprints of segments of the multimedia signal are generated and matched against fingerprints in the database. When a match is found, one or more associated actions for the segment are retrieved by the device. The trigger time point may be determined as a time point near or at the segment of the multimedia signal with the matched fingerprint. In this way, trigger markers for actions may be enabled without modifying the multimedia signal.

Claims (56)

1. A method comprising:

receiving multimedia content;

determining a time point indicating when, in the multimedia content, an action is to be executed;

deriving, by a fingerprint generation device having a processor coupled to a memory, a fingerprint from a respective segment of the multimedia content based on the respective segment corresponding to the determined time point;

storing the derived fingerprint in a database in association with the action; and

transmitting the derived fingerprint to a multimedia playback device.

2. The method of claim 1 , wherein the action to be executed comprises (i) retrieving and displaying additional information on a display, or (ii) retrieving and playing additional information via a speaker.

3. The method of claim 1 , wherein the action to be executed comprises presenting other multimedia content instead of the multimedia content for a period of time.

4. The method of claim 1 , wherein the action to be executed comprises presenting data overlaid on the multimedia content.

5. The method of claim 1 , wherein the action to be executed comprises replacing national content in the multimedia content with local content.

6. The method of claim 1 , wherein the action to be executed comprises executing a control command.

7. The method of claim 1 , wherein the action to be executed comprises preparing the multimedia playback device to receive a user input.

8. The method of claim 1 , wherein the time point coincides with an end of the respective segment of the multimedia content.

9. The method of claim 1 , wherein the time point coincides with a beginning of the respective segment of the multimedia content.

10. The method of claim 1 , wherein the time point is a predetermined time offset after a beginning of the respective segment of the multimedia content.

11. The method of claim 1 , wherein the time point is a predetermined time offset before an end of the respective segment of the multimedia content.

12. The method of claim 1 , further comprising:

sending a representation of the action to the multimedia playback device.

13. A non-transitory machine-readable storage medium having stored thereon, program instructions that when executed by a processor of a machine, cause the machine to perform operations comprising:

receiving multimedia content;

determining a time point indicating when, in the multimedia content, an action is to be executed;

deriving a fingerprint from a respective segment of the multimedia content based on the respective segment corresponding to the determined time point;

storing the derived fingerprint in a database in association with the action; and

transmitting the derived fingerprint to a multimedia playback device.

14. The non-transitory machine-readable storage medium of claim 13 , wherein the action to be executed comprises:

retrieving and displaying additional information on a display,

retrieving and playing additional information via a speaker,

presenting other multimedia content instead of the multimedia content for a period of time, or

presenting data overlaid on the multimedia content.

15. The non-transitory machine-readable storage medium of claim 13 , wherein:

the time point coincides with an end of the respective segment of the multimedia content,

the time point coincides with a beginning of the respective segment of the multimedia content,

the time point is a predetermined time offset after a beginning of the respective segment of the multimedia content, or

the time point is a predetermined time offset before an end of the respective segment of the multimedia content.

16. The non-transitory machine-readable storage medium of claim 13 , the operations further comprising:

sending a representation of the action to the multimedia playback device.

17. A system comprising:

one or more processors; and

a non-transitory computer-readable storage medium, having stored thereon program instructions that, upon execution by the one or more processors, cause performance of a set of operations comprising:

receiving multimedia content,

determining a time point indicating when, in the multimedia content, an action is to be executed,

deriving a fingerprint from a respective segment of the multimedia content based on the respective segment corresponding to the determined time point,

storing the derived fingerprint in a database in association with the action, and

transmitting the derived fingerprint to a multimedia playback device.

18. The system of claim 17 , wherein the action to be executed comprises:

retrieving and displaying additional information on a display,

retrieving and playing additional information via a speaker,

presenting other multimedia content instead of the multimedia content for a period of time, or

presenting data overlaid on the multimedia content.

19. The system of claim 17 , wherein:

the time point coincides with an end of the respective segment of the multimedia content,

the time point coincides with a beginning of the respective segment of the multimedia content,

the time point is a predetermined time offset after a beginning of the respective segment of the multimedia content, or

the time point is a predetermined time offset before an end of the respective segment of the multimedia content.

20. The system of claim 17 , wherein the set of operations further comprises:

sending a representation of the action to the multimedia playback device.

Assignments (11)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: GRACENOTE, INC.
To: ROKU, INC.
Reel/Frame 056103/0786 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2020
From: KONINKLIJKE PHILIPS ELECTRONICS N.V.
To: GRACENOTE, INC.
Reel/Frame 052142/0994 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2020
From: OOSTVEEN, JOB CORNELIS; TEN KATE, WARNER RUDOLPH THEOPHILE; DENISSEN, ADRIANUS JOHANNES MARIA; ROBERTS, DAVID KEITH
To: KONINKLIJKE PHILIPS ELECTRONICS N.V.
Reel/Frame 052142/0975 →
Priority Claims (1)
EP 03102116 · Jul 11, 2003 · regional
Continuity (10)
Continuation 16264134 · Jan 31, 2019
Continuation 16018022 · Jun 25, 2018
Continuation 15622024 · Jun 13, 2017
Continuation 15273185 · Sep 22, 2016
Continuation 15164605 · May 25, 2016
Continuation 14659044 · Mar 16, 2015
Continuation 14151879 · Jan 10, 2014
Continuation 13220267 · Aug 29, 2011
Continuation 10564297
Related Publication 20200322653A1 · Oct 8, 2020