IP Library Granted Patent US 11,386,696
Granted Patent B2
US 11,386,696 · App. 16/870,583 · Granted Jul 12, 2022

Content-modification system with fingerprint data mismatch and responsive action feature

Inventors: Andrew Laurence Ashbacher (Aptos, CA); Steven Michael Cormie (Cambridge, GB); Stefanos Christakis Liassides (Cardiff, GB)
Assignee: Roku, Inc.
G06V40/1365G06F16/535G06F16/583G06V40/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,386,696
App. No.
16/870,583
Granted
Jul 12, 2022
Kind
B2
Abstract

In one aspect, a method includes receiving query fingerprint data representing content received by a content-presentation device. The method also includes comparing at least a first portion of the query fingerprint data with at least one first reference fingerprint data set, where each first reference fingerprint data set corresponds with a respective channel. The method also includes determining that the query fingerprint data does not match any of the at least one first reference fingerprint data sets. The method also includes responsive to the determining, comparing at least a second portion of the query fingerprint data with second reference fingerprint data representing a modifiable content-segment. The method also includes detecting a match between at least the second portion of the query fingerprint data and the second reference fingerprint data. The method also includes responsive to detecting the match, performing an action that facilitates the content-presentation device performing a content-modification operation.

Claims (55)

1. A method comprising:

receiving query fingerprint data representing content received by a content-presentation device on a channel that is unidentified to a fingerprint-matching server;

making a first comparison of at least a first portion of the received query fingerprint data with at least one first reference fingerprint data set of a plurality of first reference fingerprint data sets, wherein each of the plurality of first reference fingerprint data sets corresponds with a respective known channel identifier and is generated by a content-distribution system from which the content-presentation device receives the content on the unidentified channel;

based on the first comparison, determining that the received query fingerprint data does not match any of the at least one first reference fingerprint data sets;

responsive to the determining;

determining that the channel on which the content-presentation device is receiving the content is an unidentified channel, and

making a second comparison of at least a second portion of the received query fingerprint data with second reference fingerprint data representing a modifiable content-segment, wherein the second reference fingerprint data is different from the plurality of first reference fingerprint data sets that correspond with the known channel identifiers and is generated by a computing system different from the content-distribution system;

based on the second comparison, detecting a match between at least the second portion of the received query fingerprint data and the second reference fingerprint data; and

responsive to detecting the match;

determining that the content received by the content-presentation device on the unidentified channel includes the modifiable content-segment, and

performing an action that facilitates the content-presentation device performing a content-modification operation so as to output supplemental content on the unidentified channel.

2. The method of claim 1 , wherein at least the first portion of the received query fingerprint data and at least the second portion of the received query fingerprint data are the same.

3. The method of claim 1 , further comprising:

determining a transmission time-stamp representing a time-point at which a content-distribution system transmitted the modifiable content-segment, wherein performing the action comprises transmitting the transmission time-stamp to the content-presentation device to facilitate the content-presentation device using the transmission time-stamp to determine a modification start-time at which to perform the content-modification operation.

4. The method of claim 3 , further comprising:

determining a starting frame marker of the modifiable content-segment, wherein performing the action further comprises transmitting the starting frame marker to the content-presentation device to facilitate the content-presentation device using the transmission time-stamp and the starting frame marker to determine a modification start-time at which to perform the content-modification operation.

5. The method of claim 1 , wherein performing the content-modification operation comprises performing a content-modification operation other than a content-replacement operation.

6. The method of claim 5 , wherein performing the content-modification operation comprises performing a content-overlay operation.

7. The method of claim 6 , wherein performing the action further comprises performing an action that facilitates the content-presentation device retrieving overlay content for use in connection with performing the content-overlay operation.

8. The method of claim 7 , wherein performing the action that facilitates the content-presentation device retrieving the overlay content for use in connection with performing the content-overlay operation comprises transmitting, to the content-presentation device, supplemental content selection criteria that instructs the content-presentation device to retrieve the overlay content.

9. The method of claim 1 , wherein the method is performed by a fingerprint-matching server of a content-modification system.

10. A non-transitory computer-readable storage medium, having stored thereon program instructions that, upon execution by a processor, cause performance of a set of operations comprising:

receiving query fingerprint data representing content received by a content-presentation device on a channel that is unidentified to a fingerprint-matching server;

making a first comparison of at least a first portion of the received query fingerprint data with at least one first reference fingerprint data set of a plurality of first reference fingerprint data sets, wherein each of the plurality of first reference fingerprint data sets corresponds with a respective known channel identifier and is generated by a content-distribution system from which the content-presentation device receives the content on the unidentified channel;

based on the first comparison, determining that the received query fingerprint data does not match any of the at least one first reference fingerprint data sets;

responsive to the determining;

determining that the channel on which the content-presentation device is receiving the content is an unidentified channel, and

making a second comparison of at least a second portion of the received query fingerprint data with second reference fingerprint data representing a modifiable content-segment, wherein the second reference fingerprint data is different from the plurality of first reference fingerprint data sets that correspond with the known channel identifiers and is generated by a computing system different from the content-distribution system;

based on the second comparison, detecting a match between at least the second portion of the received query fingerprint data and the second reference fingerprint data; and

responsive to detecting the match;

determining that the content received by the content-presentation device on the unidentified channel includes the modifiable content-segment, and

performing an action that facilitates the content-presentation device performing a content-modification operation so as to output supplemental content on the unidentified channel.

11. The non-transitory computer-readable storage medium of claim 10 , wherein at least the first portion of the received query fingerprint data and at least the second portion of the received query fingerprint data are the same.

12. The non-transitory computer-readable storage medium of claim 10 , the set of operations further comprising:

determining a transmission time-stamp representing a time-point at which a content-distribution system transmitted the modifiable content-segment, wherein performing the action comprises transmitting the transmission time-stamp to the content-presentation device to facilitate the content-presentation device using the transmission time-stamp to determine a modification start-time at which to perform the content-modification operation.

13. The non-transitory computer-readable storage medium of claim 10 , wherein performing the content-modification operation comprises performing a content-modification operation other than a content-replacement operation.

14. The non-transitory computer-readable storage medium of claim 13 , wherein performing the content-modification operation comprises performing a content-overlay operation.

15. The non-transitory computer-readable storage medium of claim 14 , wherein performing the action further comprises performing an action that facilitates the content-presentation device retrieving overlay content for use in connection with performing the content-overlay operation.

16. A computing system comprising:

a processor; and

a non-transitory computer-readable storage medium, having stored thereon program instructions that, upon execution by the processor, cause performance of a set of operations comprising:

receiving query fingerprint data representing content received by a content-presentation device on a channel that is unidentified to a fingerprint-matching server;

making a first comparison of at least a first portion of the received query fingerprint data with at least one first reference fingerprint data set of a plurality of first reference fingerprint data sets, wherein each of the plurality of first reference fingerprint data sets corresponds with a respective known channel identifier and is generated by a content-distribution system from which the content-presentation device received the content on the unidentified channel;

based on the first comparison, determining that the received query fingerprint data does not match any of the at least one first reference fingerprint data sets;

responsive to the determining;

determining that the channel on which the content-presentation device is receiving the content is an unidentified channel, and

making a second comparison of at least a second portion of the received query fingerprint data with second reference fingerprint data representing a modifiable content-segment, wherein the second reference fingerprint data is different from the plurality of first reference fingerprint data sets that correspond with the known channel identifiers and is generated by a computing system different from the content-distribution system;

based on the second comparison, detecting a match between at least the second portion of the received query fingerprint data and the second reference fingerprint data; and

responsive to detecting the match;

determining that the content received by the content-presentation device on the unidentified channel includes the modifiable content-segment, and

performing an action that facilitates the content-presentation device performing a content-modification operation so as to output supplemental content on the unidentified channel.

17. The computing system of claim 16 , wherein at least the first portion of the received query fingerprint data and at least the second portion of the received query fingerprint data are the same.

18. The computing system of claim 16 , wherein performing the content-modification operation comprises performing a content-modification operation other than a content-replacement operation.

19. The computing system of claim 18 , wherein performing the content-modification operation comprises performing a content-overlay operation.

20. The computing system of claim 19 , wherein performing the action further comprises performing an action that facilitates the content-presentation device retrieving overlay content for use in connection with performing the content-overlay operation.

Assignments (10)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2020
From: ASHBACHER, ANDREW LAURENCE; CORMIE, STEVEN MICHAEL; LIASSIDES, STEFANOS CHRISTAKIS
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 052623/0582 →