IP Library Granted Patent US 11,212,560
Granted Patent B2
US 11,212,560 · App. 15/929,216 · Granted Dec 28, 2021

Use of steganographically-encoded time information as basis to establish a time offset, to facilitate taking content-related action

Inventors: Benjamin J. Ropke (Rochelle, NY); John S. Stavropoulos (Edison, NJ); Shashank C. Merchant (Sunnyvale, CA)
Assignee: The Nielsen Company (US), LLC
H04N21/23424G06T1/0085H04N21/23892H04N21/44008H04N21/44016H04N21/812H04N21/8358H04N21/8456H04N21/8547
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,212,560
App. No.
15/929,216
Granted
Dec 28, 2021
Kind
B2
Abstract

A method and system for using steganographically-encoded time information as a basis to control when a client carries out a content-related action. The client processes for presentation one or more linear media streams, each defining a respective time sequence of frames of media content, and at least one of the streams being steganographically encoded with at least one watermark at a respective time-point within the stream, the watermark encoding a timestamp of the respective time-point within the stream according to a server clock. The client extracts the timestamp from the watermark and computes a time offset based on a difference between the extracted timestamp and a current time according to a client clock. And the client uses the computed time offset as a basis to determine when the client should carry out a content-related action in a given one of the one or more linear media streams.

Claims (47)

1. A method comprising:

processing by a client one or more linear media streams for presentation, each of the one or more linear media streams defining a respective time sequence of frames of media content, and at least one of the linear media streams being steganographically encoded with at least one watermark at a respective time-point within the linear media stream, the watermark encoding a timestamp of the respective time-point within the linear media stream according to a server clock;

extracting by the client the timestamp from the watermark, and computing by the client a time offset based on a difference between the extracted timestamp and a current time according to a client clock;

receiving by the client from a server a directive for the client to carry out a content-related action in a given one of the one or more linear media streams, the directive specifying an action time point when the client should carry out the content-related action, the action time point being defined according to the server clock; and

using by the client the computed time offset as a basis to determine when the client should carry out the content-related action in the given linear media stream,

wherein the content-related action comprises dynamic content modification, and

wherein using by the client the computed time offset as a basis to determine when the client should carry out the content-related action in the given linear media stream comprises applying by the client the computed time offset as a basis to translate the specified action time point into an associated client time point according to the client clock.

2. The method of claim 1 , wherein computing the time offset based on the difference between the extracted timestamp and the current time according to the client clock comprises computing the time offset as the difference.

3. The method of claim 1 , wherein the at least one linear media stream is steganographically encoded with multiple watermarks each at a respective media-stream time-point and each encoding a respective timestamp of the respective media-stream time-point according to the server clock, the method further comprising:

extracting by the client from each of the multiple watermarks the respective timestamp, and

computing by the client multiple time offsets including, respectively for each extracted respective timestamp, a respective time offset between the extracted respective timestamp and a respective current time according to the client clock,

wherein computing by the client the time offset based on a difference between the extracted timestamp and the current time according to the client clock comprises computing by the client a statistical representation of the multiple computed time offsets.

4. The method of claim 3 , wherein the statistical measure comprises an average.

5. The method of claim 1 , wherein the content-related action comprises dynamic ad replacement.

6. The method of claim 1 , wherein the applying the computed time offset as the basis to translate the specified action time point into the associated client time point according to the client clock comprises adding the computed time offset to the specified action time point.

7. The method of claim 1 , wherein processing by the client the one or more linear media streams for presentation comprises receiving each of the one or more linear media streams distributed by a content distribution system, wherein the server clock defines time according to the content distribution system.

8. The method of claim 1 , further comprising:

generating, by the client, query fingerprints representing the given linear media stream; and

transmitting, from the client to the server, the generated query fingerprints to enable the server to identify the given linear media stream.

9. A method comprising:

receiving by a client an ongoing media stream that includes watermarks steganographically encoding server-timestamps of portions of the media stream from a time perspective of a sever;

for each of one or more of the steganographically encoded watermarks in the media stream being received by the client, extracting by the client the server-timestamp encoded by the watermark and comparing by the client the extracted server-timestamp with a respective client-time from a time perspective of the client, to compute a time offset useable by the client as a basis to determine when the client should carry out a content-related action;

receiving by the client from a server a directive for the client to carry out a content-related action in the media stream, the directive specifying an action time point when the client should carry out the content-related action, the action time point being from the time perspective of the server; and

using by the client the computed time offset as a basis to determine when the client should carry out the content-related action in the media stream, wherein the content-related action comprises dynamic content modification, and wherein using by the client the computed time offset as a basis to determine when the client should carry out the content-related action in the media stream comprises applying by the client the computed time offset as a basis to translate the specified action time point into an associated client time point from the time perspective of the client.

10. The method of claim 9 , wherein the content-related action comprises dynamic ad replacement.

11. A media client comprising:

a content input interface;

a content presentation interface;

a network communication interface;

a processing unit;

non-transitory data storage; and

program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations including:

processing one or more linear media streams for presentation by the media client, each of the one or more linear media streams defining a respective time sequence of frames of media content, and at least one of the linear media streams being steganographically encoded with at least one watermark at a respective time-point within the linear media stream, the watermark encoding a timestamp of the respective time-point within the linear media stream according to a server clock,

extracting the timestamp from the watermark, and computing a time offset based on a difference between the extracted timestamp and a current time according to a media-client clock,

receiving from a server a directive for the media client to carry out a content-related action in a given one of the one or more linear media streams, the directive specifying an action time point when the media client should carry out the content-related action, the action time point being defined according to the server clock, and

using the computed time offset as a basis to determine when the media client should carry out the content-related action in the given linear media stream,

wherein the content-related action comprises dynamic content modification, and

wherein using the computed time offset as a basis to determine when the media client should carry out the content-related action in the given linear media stream comprises applying the computed time offset as a basis to translate the specified action time point into an associated client time point according to the media-client clock.

12. The media client of claim 11 , wherein the media client is a television.

13. The media client of claim 11 , wherein computing the time offset based on the difference between the extracted timestamp and the current time according to the media-client clock comprises computing the time offset as the difference.

14. The media client of claim 11 , wherein the at least one linear media stream is steganographically encoded with multiple watermarks each at a respective media-stream time-point and each encoding a respective timestamp of the respective media-stream time-point according to the server clock, the operations further including:

extracting from each of the multiple watermarks the respective timestamp, and

computing multiple time offsets including, respectively for each extracted respective timestamp, a respective time offset between the extracted respective timestamp and a respective current time according to the media-client clock,

wherein computing the time offset based on a difference between the extracted timestamp and the current time according to the media-client clock comprises computing by the client a statistical representation of the multiple computed time offsets.

15. The media client of claim 11 , wherein the content-related action comprises dynamic ad replacement.

16. The media client of claim 11 , wherein processing the one or more linear media streams for presentation by the media client comprises receiving each of the one or more linear media streams distributed by a content distribution system, wherein the server clock defines time according to the content distribution system.

17. The media client of claim 15 , wherein the dynamic content modification is responsive to fingerprint-based automatic content recognition.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: GRACENOTE, INC.; A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 28, 2021
From: ROPKE, BENJAMIN J.; STAVROPOULOS, JOHN S.; MERCHANT, SHASHANK C.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 057948/0310 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →