IP Library Granted Patent US 10,904,587
Granted Patent B2
US 10,904,587 · App. 16/166,971 · Granted Jan 26, 2021

Establishment and use of time mapping based on interpolation using low-rate fingerprinting, to help facilitate frame-accurate content revision

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,904,587
App. No.
16/166,971
Granted
Jan 26, 2021
Kind
B2
Abstract

A computing system identifies multiple matching points between (i) query fingerprints representing a media stream being received by a client and (ii) reference fingerprints, each identified matching point defining a respective match between a query fingerprint timestamped with client time defined according to a clock of the client and a reference fingerprint timestamped with true time defined according to a timeline within a known media stream. Further, the computing system performs linear regression based on the timestamps of the matching points, to establish a mapping between true time and client time. The computing system then uses the mapping to determine a client-time point at which the client should perform a content revision or other action with respect to the media stream being received by the client. And the computing system causes the client to perform the content revision or other action at the determined client-time point.

Claims (43)

1. A method comprising:

receiving, by a computing system, query fingerprints representing a media stream that is being received by a client, the query fingerprints being timestamped with client time according to a clock of the client;

comparing by the computing system the received query fingerprints with reference fingerprints representing a known media stream, the reference fingerprints being timestamped with true time according to a timeline within the known media stream;

detecting, by the computing system, based on the comparing, a plurality of matching points each defining a match between a query fingerprint having an associated client-time timestamp and a reference fingerprint having an associated true-time timestamp;

performing by the computing system linear regression based on the detected matching points, the linear regression establishing a mapping between true time and client time;

applying by the computing system the established mapping as a basis to determine a client-time point at which the client should perform an action with respect to the media stream being received by the client; and

causing the client to perform, at the determined client-time point, the action,

wherein each matching point is based on a comparison between (i) a respective bundle of the query fingerprints and (ii) the reference fingerprints, and wherein the method further comprises detecting that the query fingerprints of a given bundle of the query fingerprints match each other and, responsive to the detecting that the query fingerprints of the given bundle match each other, excluding the given bundle from use to establish a matching point for the linear regression.

2. The method of claim 1 , wherein the action with respect to the media stream being received by the client comprises a content revision.

3. The method of claim 2 , wherein the media stream being received by the client comprises an ad pod defining a sequence of ads, and wherein the content revision comprises replacing an ad of the sequence with replacement media content.

4. The method of claim 1 , wherein the action with respect to the media stream being received by the client comprises preparation for a content revision.

5. The method of claim 1 , wherein applying the established mapping as a basis to determine a client-time point at which the client should perform the action with respect to the media stream being received by the client comprises using the established mapping as a basis to map a predefined true-time point at which the action should be performed with respect to the known media stream to a corresponding client-time point at which the client should perform the action with respect to the media stream being received by the client.

6. The method of claim 1 , wherein the computing system is implemented by a server in network communication with the client.

7. The method of claim 1 , wherein the given bundle of query fingerprints is a bundle of consecutive query fingerprints.

8. The method of claim 1 , further comprising using by the computing system one or more of the detected matching points as a basis to determine that the media stream being received by the client is the known media stream,

wherein the performing of linear regression, the applying of the mapping, and the causing of the client to perform the action at the determined client-time point are collectively responsive to at least the determining that the media stream being received by the client is the known media stream.

9. A computing system comprising:

a network communication interface through which to transmit and receive;

a processing unit;

non-transitory data storage; and

program instructions stored in the non-transitory data storage and executable by the processing unit to cause the computing system to carry out operations including:

identifying a plurality of matching points between (i) query fingerprints representing a media stream being received by a media client and (ii) reference fingerprints, wherein each identified matching point defines a respective match between (i) a query fingerprint that is timestamped with client time defined according to a clock of the media client and (ii) a reference fingerprint that is timestamped with true time defined according to a timeline within a known media stream,

performing linear regression based on the timestamps of the identified plurality of matching points, the linear regression establishing a mapping between true time and client time,

using the established mapping as a basis to determine a client-time point at which the media client should perform a content revision in the media stream being received by the media client, and

causing the media client to perform, at the determined client-time point, the content revision,

wherein each matching point is based on a comparison between (i) a respective bundle of the query fingerprints and (ii) the reference fingerprints, the operations further including detecting that the query fingerprints of a given bundle of the query fingerprints match each other and, responsive to the detecting that the query fingerprints of the given bundle match each other, excluding the given bundle from use to identify a matching point for the linear regression.

10. The computing system of claim 9 , wherein using the established mapping as a basis to determine the client-time point at which the media client should perform the content revision in the media stream being received by the media client comprises using the established mapping as a basis to map a predefined true-time point at which the content revision should be performed in the known media stream to a corresponding client-time point at which the media client should perform the content revision in the media stream being received by the media client.

11. The computing system of claim 9 , wherein the content revision comprises content replacement.

12. The computing system of claim 11 , wherein the media stream being received by the media client comprises an ad pod defining a sequence of ads, and wherein the content revision comprises replacing an ad of the sequence with replacement media content.

13. The computing system of claim 9 , wherein the operations further include receiving the query fingerprints from the media client, and wherein causing the media client to perform the content revision at the determined client-time point comprises transmitting to the media client a directive that causes the media client to perform the content revision at the determined client-time point.

14. The computing system of claim 9 , wherein the given bundle of query fingerprints is a bundle of consecutive query fingerprints.

15. The computing system of claim 9 , wherein the operations further include using one or more of the identified matching points as a basis to determine that the media stream being received by the media client is the known media stream,

wherein the performing of linear regression, the using of the mapping, and the causing of the media client to perform the content revision at the determined client-time point are collectively responsive to at least the determining that the media stream being received by the media client is the known media stream.

16. A non-transitory computer readable medium storing instructions executable by a processing unit to carry out operations comprising:

identifying a plurality of matching points between (i) query fingerprints representing a media stream being received by a client and (ii) reference fingerprints, wherein each identified matching point defines a respective match between (i) a query fingerprint that is timestamped with client time defined according to a clock of the client and (ii) a reference fingerprint that is timestamped with true time defined according to a timeline within a known media stream;

performing linear regression based on the timestamps of the identified plurality of matching points, the linear regression establishing a mapping between true time and client time;

using the established mapping as a basis to determine a client-time point at which the client should perform a content revision in the media stream being received by the client; and

causing the client to perform, at the determined client-time point, the content revision,

wherein each matching point is based on a comparison between (i) a respective bundle of the query fingerprints and (ii) the reference fingerprints, the operations further including detecting that the query fingerprints of a given bundle of the query fingerprints match each other and, responsive to the detecting that the query fingerprints of the given bundle match each other, excluding the given bundle from use to identify a matching point for the linear regression.

17. The non-transitory computer readable medium of claim 16 , wherein using the established mapping as a basis to determine the client-time point at which the client should perform the content revision in the media stream being received by the client comprises using the established mapping as a basis to map a predefined true-time point at which the content revision should be performed in the known media stream to a corresponding client-time point at which the client should perform the content revision in the media stream being received by the client.

18. The non-transitory computer readable medium of claim 16 , wherein the given bundle of query fingerprints is a bundle of consecutive query.

19. The non-transitory computer readable medium of claim 16 , wherein causing the client to perform the content revision at the determined client-time point comprises outputting for transmission to the client a content-revision directive indicating the determined client-time point.

20. The non-transitory computer readable medium of claim 16 , wherein the media stream being received by the client comprises an ad pod defining a sequence of ads, and wherein the content revision comprises replacing an ad of the sequence with replacement media content.

Assignments (10)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: GRACENOTE, INC.; A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: GRACENOTE, INC.
To: ROKU, INC.
Reel/Frame 056103/0786 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2018
From: DUNKER, PETER; CREMER, MARKUS K.; MERCHANT, SHASHANK C.; THIELEN, KURT R.
To: GRACENOTE, INC.
Reel/Frame 047387/0319 →