IP Library Granted Patent US 12,389,088
Granted Patent B2
US 12,389,088 · App. 18/661,845 · Granted Aug 12, 2025

Use of watermarking to trigger fingerprint-related action

Inventors: Shashank C. Merchant (Sunnyvale, CA); John S. Stavropoulos (Edison, NJ); Benjamin J. Ropke (Rochelle, NY); Steven M. Cormie (Cambridge, GB)
Assignee: ROKU, INC.
H04N21/8358G06F21/16H04N21/2353H04N21/44008H04N21/4415
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,389,088
App. No.
18/661,845
Granted
Aug 12, 2025
Kind
B2
Abstract

Use of watermarking in a media stream as a trigger for carrying out a fingerprint-related action, such as starting to generate fingerprints of the media stream, increasing a rate of generation of fingerprints of the media stream, starting to report fingerprints of the media stream, requesting fingerprints, or engaging in a fingerprint comparison. In an implementation, a media client that is processing the media stream could detect the watermark in the media stream and could responsively carry out or cause to be carried out a fingerprint-related action. Or an entity in a media-distribution path along which the media stream flows to a media client could detect the watermark in the media stream and could responsively carry out or cause to be carried out a fingerprint-related action. Further, the watermark could be inserted into the media stream in the first place based on transport-stream carried metadata, among other possibilities.

Claims (55)

1. A method comprising:

extracting, by a media client from an out-of-band signal, a watermark payload including an advertisement identifier of an upcoming advertisement in a media stream being processed by the media client, wherein the watermark payload comprises a time offset;

establishing a synchronous lock between a current time according to a clock of the media client and the time offset;

receiving, by the media client, digital reference fingerprints representing the upcoming advertisement corresponding to the advertisement identifier;

transitioning from generating digital query fingerprints representing the media stream at a first non-zero fingerprint-generation rate to a second fingerprint-generation rate that is greater than the first non-zero fingerprint-generation rate; and

comparing, by the media client, digital query fingerprints generated at the second fingerprint-generation rate with the digital reference fingerprints to determine when in the media stream the upcoming advertisement begins.

2. The method of claim 1 , wherein the digital reference fingerprints also represent media content preceding the upcoming advertisement.

3. The method of claim 1 , wherein the media client comprises a streaming media receiver or a television.

4. The method of claim 1 , further comprising:

engaging in content modification of a modifiable-content segment of the media stream corresponding to the upcoming advertisement;

generating digital query fingerprints of the modifiable-content segment during the content modification; and

comparing the digital query fingerprints of the modifiable-content segment with the digital reference fingerprints.

5. The method of claim 4 , further comprising:

determining, based on the comparing, that the digital query fingerprints of the modifiable-content segment do not match the digital reference fingerprints; and

based at least on the determination of a mismatch, stopping the content modification.

6. The method of claim 5 , further comprising reverting to play the modifiable-content segment of the media stream.

7. The method of claim 1 , wherein the watermark payload comprises an absolute time, and the method further comprises establishing a time mapping to translate the absolute time to a current time according to a clock of the media client.

8. The method of claim 1 , further comprising:

receiving, via Internet Protocol (IP) messaging, a message that correlates a reference code in the watermark payload with content-revision characteristics; and

performing the content-revision characteristics on the media stream.

9. A media client comprising:

a processing unit; and

non-transitory data storage coupled to the processing unit, wherein the processing unit is configured to:

extract, from an out-of-band signal, a watermark payload including an advertisement identifier of an upcoming advertisement in a media stream being processed by the media client, wherein the watermark payload comprises an absolute time;

establish a time mapping to translate the absolute time to a current time according to a clock of the media client;

receive digital reference fingerprints representing the upcoming advertisement corresponding to the advertisement identifier;

transition from generating digital query fingerprints representing the media stream at a first non-zero fingerprint-generation rate to a second fingerprint-generation rate that is greater than the first non-zero fingerprint-generation rate; and

compare digital query fingerprints generated at the second fingerprint-generation rate with the digital reference fingerprints to determine when in the media stream the upcoming advertisement begins.

10. The media client of claim 9 , wherein the digital reference fingerprints also represent media content preceding the upcoming advertisement.

11. The media client of claim 9 , wherein the media client comprises a streaming media receiver or a television.

12. The media client of claim 9 , wherein the processing unit is further configured to:

engage in content modification of a modifiable-content segment of the media stream corresponding to the upcoming advertisement;

generate digital query fingerprints of the modifiable-content segment during the content modification; and

compare the digital query fingerprints of the modifiable-content segment with the digital reference fingerprints.

13. The media client of claim 12 , wherein the processing unit is further configured to:

determine, based on the comparison, that the digital query fingerprints of the modifiable-content segment do not match the digital reference fingerprints; and

based at least on the determination of a mismatch, stop the content modification.

14. The media client of claim 13 , wherein the processing unit is further configured to revert to play the modifiable-content segment of the media stream.

15. A non-transitory computer-readable medium comprising instructions that when executed by a processing unit of a media client, cause the media client to perform operations, the operations comprising:

extracting, from an out-of-band signal, a watermark payload including an advertisement identifier of an upcoming advertisement in a media stream being processed by the media client;

receiving digital reference fingerprints representing the upcoming advertisement corresponding to the advertisement identifier;

transitioning from generating digital query fingerprints representing the media stream at a first non-zero fingerprint-generation rate to a second fingerprint-generation rate that is greater than the first non-zero fingerprint-generation rate;

comparing digital query fingerprints generated at the second fingerprint-generation rate with the digital reference fingerprints to determine when in the media stream the upcoming advertisement begins;

receiving, via Internet Protocol (IP) messaging, a message that correlates a reference code in the watermark payload with content-revision characteristics; and

performing the content-revision characteristics on the media stream.

16. The non-transitory computer-readable medium of claim 15 , wherein the digital reference fingerprints also represent media content preceding the upcoming advertisement.

17. The non-transitory computer-readable medium of claim 15 , wherein the media client comprises a streaming media receiver or a television.

18. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:

engaging in content modification of a modifiable-content segment of the media stream corresponding to the upcoming advertisement;

generating digital query fingerprints of the modifiable-content segment during the content modification; and

comparing the digital query fingerprints of the modifiable-content segment with the digital reference fingerprints.

19. The non-transitory computer-readable medium of claim 18 , wherein the operations further comprise:

determining, based on the comparing, that the digital query fingerprints of the modifiable-content segment do not match the digital reference fingerprints; and

based at least on the determination of a mismatch, stopping the content modification.

20. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise reverting to play the modifiable-content segment of the media stream.

Assignments (3)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2024
From: MERCHANT, SHASHANK C.; STAVROPOULOS, JOHN S.; ROPKE, BENJAMIN J.; CORMIE, STEVEN M.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 067449/0471 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2024
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 067456/0396 →
Continuity (7)
Continuation 17648638 · Jan 21, 2022
Continuation 15929219 · Feb 6, 2020
Provisional Application 62809360 · Feb 22, 2019
Provisional Application 62865613 · Jun 24, 2019
Provisional Application 62882217 · Aug 2, 2019
Provisional Application 62882202 · Aug 2, 2019
Related Publication 20240298079A1 · Sep 5, 2024
References Cited (41)
US 5450490A · Jensen et al. · 1995 [cited by applicant]
US 5574962A · Fardeau et al. · 1996 [cited by applicant]
US 5581800A · Fardeau et al. · 1996 [cited by applicant]
US 5764763A · Jensen et al. · 1998 [cited by applicant]
US 5787334A · Fardeau et al. · 1998 [cited by applicant]
US 6871180B1 · Neuhauser et al. · 2005 [cited by applicant]
US 8359205B2 · Srinivasan et al. · 2013 [cited by applicant]
US 8369972B2 · Topchy et al. · 2013 [cited by applicant]
US 10506275B1 · Thielen · 2019 [cited by examiner]
US 10735782B2 · Gordon · 2020 [cited by applicant]
US 20020162118A1 · Levy et al. · 2002 [cited by applicant]
US 20100223062A1 · Srinivasan et al. · 2010 [cited by applicant]
US 20140075466A1 · Zhao · 2014 [cited by applicant]
US 20140114455A1 · Larsen et al. · 2014 [cited by applicant]
US 20140201769A1 · Neumeier · 2014 [cited by examiner]
US 20140245018A1 · Mooneyham et al. · 2014 [cited by applicant]
US 20150106505A1 · Ramaswamy et al. · 2015 [cited by applicant]
US 20150117700A1 · Im et al. · 2015 [cited by applicant]
US 20150249870A1 · Sato · 2015 [cited by applicant]
US 20150326952A1 · Estevam de Britto et al. · 2015 [cited by applicant]
US 20160127755A1 · Woods · 2016 [cited by examiner]
US 20180063558A1 · Stefanidis · 2018 [cited by examiner]
US 20180192119A1 · Stigall · 2018 [cited by examiner]
US 20180234728A1 · Hwang et al. · 2018 [cited by applicant]
US 20180302670A1 · Lee et al. · 2018 [cited by applicant]
US 20180343481A1 · Loheide · 2018 [cited by examiner]
US 20180367868A1 · Banger · 2018 [cited by examiner]
US 20190028778A1 · Liassides et al. · 2019 [cited by applicant]
US 20190082209A1 · Moon et al. · 2019 [cited by applicant]
US 20190082231A1 · Chen · 2019 [cited by applicant]
US 20190132652A1 · Zhao et al. · 2019 [cited by applicant]
US 20200275168A1 · Merchant et al. · 2020 [cited by applicant]
US 20200404371A1 · Merchant · 2020 [cited by examiner]
US 20210133814A1 · Thielen · 2021 [cited by examiner]
US 20210344976A1 · Sharifi · 2021 [cited by examiner]
US 20220109715A1 · Cormie · 2022 [cited by examiner]
US 20220150601A1 · Merchant et al. · 2022 [cited by applicant]
WO WO2014007571 · 2014 [cited by applicant]
WO WO2014007575 · 2014 [cited by applicant]
International Search Report and Written Opinion from International Application No. PCT/US2020/037898, dated Sep. 25, 2020. [cited by applicant]
International Preliminary Report on Patentability from International Application No. PCT/US2020/037898, dated Dec. 28, 2021. [cited by applicant]