IP Library Granted Patent US 12,015,833
Granted Patent B2
US 12,015,833 · App. 17/648,638 · Granted Jun 18, 2024

Use of watermarking to trigger fingerprint-related action

Inventors: Shashank C. Merchant (Sunnyvale, CA); John S. Stavropoulos (Edison, NJ); Benjamin J. Ropke (Rochelle, NY); Steven M. Cormie (Cambridge, GB)
Assignee: Roku, Inc.
H04N21/8358G06F21/16H04N21/2353H04N21/44008H04N21/4415
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,015,833
App. No.
17/648,638
Granted
Jun 18, 2024
Kind
B2
Abstract

Use of watermarking in a media stream as a trigger for carrying out a fingerprint-related action, such as starting to generate fingerprints of the media stream, increasing a rate of generation of fingerprints of the media stream, starting to report fingerprints of the media stream, requesting fingerprints, or engaging in a fingerprint comparison. In an implementation, a media client that is processing the media stream could detect the watermark in the media stream and could responsively carry out or cause to be carried out a fingerprint-related action. Or an entity in a media-distribution path along which the media stream flows to a media client could detect the watermark in the media stream and could responsively carry out or cause to be carried out a fingerprint-related action. Further, the watermark could be inserted into the media stream in the first place based on transport-stream carried metadata, among other possibilities.

Claims (48)

1. A method comprising:

extracting, by a media client from an out-of-band signal, a watermark payload including an advertisement identifier of an upcoming advertisement in a media stream being processed by the media client, wherein the watermark payload is based on transport stream metadata associated with the media stream;

receiving, by the media client, digital reference fingerprints representing the upcoming advertisement corresponding to the advertisement identifier;

transitioning from generating digital query fingerprints representing the media stream at a first non-zero fingerprint-generation rate to a second fingerprint-generation rate that is greater than the first non-zero fingerprint-generation rate; and

comparing, by the media client, digital query fingerprints generated at the second fingerprint-generation rate with the digital reference fingerprints to determine when in the media stream the upcoming advertisement begins.

2. The method of claim 1 , wherein the digital reference fingerprints also represent the media content preceding the upcoming advertisement.

3. The method of claim 1 , wherein the media client comprises a streaming media receiver or a television.

4. The method of claim 1 , further comprising:

engaging in content modification of a modifiable-content segment of the media stream corresponding to the upcoming advertisement;

generating digital query fingerprints of the modifiable-content segment during the content modification; and

comparing the digital query fingerprints of the modifiable-content segment with the digital reference fingerprints.

5. The method of claim 4 , further comprising:

determining, based on the comparing, that the digital query fingerprints of the modifiable-content segment do not match the digital reference fingerprints; and

based at least on the mismatch, stopping the content modification.

6. The method of claim 5 , further comprising reverting to play the modifiable-content segment of the media stream.

7. The method of claim 1 , wherein the watermark payload comprises a time offset, and the method further comprises establishing a synchronous lock between a current time according to a clock of the media client and the time offset.

8. The method of claim 1 , wherein the watermark payload comprises an absolute time, and the method further comprises establishing a time mapping to translate the absolute time to a current time according to a clock of the media client.

9. The method of claim 1 , further comprising:

receiving, via Internet Protocol (IP) messaging, a message that correlates a reference code in the watermark payload with content-revision characteristics; and

performing the content-revision characteristics on the media stream.

10. A media client comprising:

a processing unit; and

non-transitory data storage coupled to the processing unit, wherein the processing unit is configured to:

extract, from an out-of-band signal, a watermark payload including an advertisement identifier of an upcoming advertisement in a media stream being processed by the media client, wherein the watermark payload is based on transport stream metadata associated with the media stream;

receive digital reference fingerprints representing the upcoming advertisement corresponding to the advertisement identifier;

transition from generating digital query fingerprints representing the media stream at a first non-zero fingerprint-generation rate to a second fingerprint-generation rate that is greater than the first non-zero fingerprint-generation rate; and

compare digital query fingerprints generated at the second fingerprint-generation rate with the digital reference fingerprints to determine when in the media stream the upcoming advertisement begins.

11. The media client of claim 10 , wherein the digital reference fingerprints also represent the media content preceding the upcoming advertisement.

12. The media client of claim 10 , wherein the media client comprises a streaming media receiver or a television.

13. The media client of claim 10 , wherein the watermark payload comprises a time offset, the processing unit is further configured to establish a synchronous lock between a current time according to a clock of the media client and the time offset.

14. The media client of claim 10 , wherein the watermark payload comprises an absolute time, the processing unit is further configured to establish a time mapping to translate the absolute time to a current time according to a clock of the media client.

15. The media client of claim 10 , wherein the processing unit is further configured to:

receive, via Internet Protocol (IP) messaging, a message that correlates a reference code in the watermark payload with content-revision characteristics; and

perform the content-revision characteristics on the media stream.

16. A non-transitory computer-readable medium comprising instructions that when executed by a processing unit of a media client, cause the media client to perform operations, the operations comprising:

extracting, from an out-of-band signal, a watermark payload including an advertisement identifier of an upcoming advertisement in a media stream being processed by the media client, wherein the watermark payload is based on transport stream metadata associated with the media stream;

receiving digital reference fingerprints representing the upcoming advertisement corresponding to the advertisement identifier;

transitioning from generating digital query fingerprints representing the media stream at a first non-zero fingerprint-generation rate to a second fingerprint-generation rate that is greater than the first non-zero fingerprint-generation rate; and

comparing digital query fingerprints generated at the second fingerprint-generation rate with the digital reference fingerprints to determine when in the media stream the upcoming advertisement begins.

17. The non-transitory computer-readable medium of claim 16 , wherein the digital reference fingerprints also represent the media content preceding the upcoming advertisement.

18. The non-transitory computer-readable medium of claim 16 , further comprising:

engaging in content modification of a modifiable-content segment of the media stream corresponding to the upcoming advertisement;

generating digital query fingerprints of the modifiable-content segment during the content modification; and

comparing the digital query fingerprints of the modifiable-content segment with the digital reference fingerprints.

19. The non-transitory computer-readable medium of claim 18 , further comprising:

determining, based on the comparing, that the digital query fingerprints of the content-modifiable segment do not match the digital reference fingerprints; and

based at least on the mismatch, stopping the content modification.

20. The non-transitory computer-readable medium of claim 19 , further comprising reverting to play the modifiable-content segment of the media stream.

Assignments (3)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2022
From: MERCHANT, SHASHANK; STAVROPOULOS, JOHN S.; ROPKE, BENJAMIN J.; CORMIE, STEVEN M.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 058729/0291 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2022
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 058807/0394 →
Continuity (6)
Continuation 15929219 · Feb 6, 2020
Provisional Application 62882217 · Aug 2, 2019
Provisional Application 62882202 · Aug 2, 2019
Provisional Application 62865613 · Jun 24, 2019
Provisional Application 62809360 · Feb 22, 2019
Related Publication 20220150601A1 · May 12, 2022