IP Library Granted Patent US 11,792,254
Granted Patent B2
US 11,792,254 · App. 17/644,183 · Granted Oct 17, 2023

Use of in-band metadata as basis to access reference fingerprints to facilitate content-related action

Inventors: Steven M. Cormie (Cambridge, GB); Brian Jentz (Buxton, GB)
Assignee: Roku, Inc.
H04L65/764H04L65/61H04L67/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,792,254
App. No.
17/644,183
Granted
Oct 17, 2023
Kind
B2
Abstract

As a media client receives a media stream for presentation, the media client could receive, in-band with the media stream, metadata identifying the media stream, such as a media-stream identifier encoded as a watermark in the media stream or carried together with the media stream in a packet-based transport stream, among other possibilities. And while the media client is processing the media stream for presentation, the media client could then convey the received metadata out-of-band to a server to enable the server to readily obtain reference fingerprints representing the media stream and to thereby facilitate, based on the reference fingerprints, a content-related action such as dynamic content modification for instance.

Claims (36)

1. A system comprising:

a processing unit;

non-transitory data storage; and

program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations including:

while a media client is processing a media stream for presentation, receiving from the media client an out-of-band signal providing an identifier of the media stream that the media client extracted from in-band metadata received by the media client with the media stream,

using the received identifier of the media stream as a basis to obtain, based on a lookup keyed to the identifier, reference fingerprints representing the media stream, without a need to search for reference fingerprints that match query fingerprints representing the media stream being processed by the media client, wherein using the received identifier as a basis to obtain the reference fingerprints based on the lookup keyed to the identifier comprises referring to predefined reference data that associates various identifiers with respective reference fingerprints, to determine, based on the reference data, the reference fingerprints with which the identifier is associated,

using the obtained reference fingerprints as a basis to determine a point in the media stream when the media client should perform a content-related action, and

causing the media client to perform, at the determined point, the content-related action.

2. The system of claim 1 , wherein the content-related action comprises dynamic ad insertion.

3. The system of claim 1 , wherein the identifier of the media stream, that the media client extracted from in-band metadata received by the media client with the media stream, comprises an identifier that the media client extracted from watermarking in the media stream.

4. The system of claim 1 , wherein the identifier of the media stream, that the media client extracted from in-band metadata received by the media client with the media stream, comprises an identifier that the media client extracted from a transport stream that carries the media stream to the media client.

5. The system of claim 1 , wherein the out-of-band signaling comprises signaling over a broadband network connection.

6. The system of claim 1 , wherein the processing of the media stream for presentation comprises at least one of presenting content of the media stream on a user interface or outputting the media stream for presentation by an associated device.

7. The system of claim 1 , wherein the causing the media client to perform, at the determined point, the content-related action comprises outputting for transmission to the media client a directive for the media client to perform the content-related action at the determined point in the media stream.

8. The system of claim 1 , wherein the operations additionally include:

using the obtained reference fingerprints as a basis to establish synchronous lock that maps reference time to client time, wherein establishing the synchronous lock enables translating a reference time of the content-related action to a client time of the content-related action.

9. The system of claim 1 , wherein the operations additionally include:

performing fingerprint comparison between a replaceable content segment and the obtained reference fingerprints to detect presence of the replaceable content segment in the media stream.

10. The system of claim 1 , wherein the media stream comprises a television channel.

11. Non-transitory data storage embodying program instructions executable by one or more processors to cause a media client to carry out operations including:

receiving a media stream, and receiving, as in-band metadata with the media stream, an identifier of the media stream;

processing the media stream for presentation; and

while processing the media stream for presentation, transmitting to a server, in out-of-band signaling, the identifier of the media stream received as the in-band metadata with the media stream, to enable the server to obtain, based on a lookup keyed to the identifier, reference fingerprints representing the media stream without a need for the server to search for reference fingerprints that match query fingerprints representing the media stream being processed for presentation, wherein obtaining the reference fingerprints based on the lookup keyed to the identifier comprises referring to predefined reference data that associates various identifiers with respective reference fingerprints, to determine, based on the reference data, the reference fingerprints with which the identifier is associated.

12. The non-transitory data storage of claim 11 , wherein the operations further include:

receiving from the server, a directive to perform a content-related action at a particular time in the media stream, wherein the particular time is established based on the obtained reference fingerprints.

13. The non-transitory data storage of claim 12 , wherein the content-related action comprises dynamic ad insertion.

14. The non-transitory data storage of claim 12 , wherein enabling the server to obtain the reference fingerprints representing the media stream enables establishment of synchronous lock that maps reference time to client time, wherein establishing of the synchronous lock enables translation of a reference time of the content-related action to a client time of the content-related action.

15. The non-transitory data storage of claim 11 , wherein enabling the server to obtain the reference fingerprints representing the media stream enables the server to perform fingerprint comparison between a replaceable content segment and the obtained reference fingerprints to detect a presence of the replaceable content segment in the media stream.

16. The non-transitory data storage of claim 11 , wherein receiving the identifier as in-band metadata with the media stream comprises receiving the identifier steganographically encoded as a watermark in the media stream or receiving the identifier carried in a transport stream that carries the media stream.

17. The non-transitory data storage of claim 11 , wherein the out-of-band signaling comprises signaling over a broadband network connection.

18. The non-transitory data storage of claim 11 , wherein the processing of the media stream for presentation comprises at least one of presenting content of the media stream on a user interface or outputting the media stream for presentation by an associated device.

19. The non-transitory data storage of claim 11 , wherein the media stream comprises a television channel.

20. A method comprising:

receiving by a media client a media stream, and receiving by the media client, as in-band metadata with the media stream, an identifier of the media stream;

processing, by the media client, the media stream for presentation; and

while processing, by the media client, the media stream for presentation, transmitting by the media client to a server, in out-of-band signaling, the identifier of the media stream received by the media client as in-band metadata with the media stream, to enable the server to obtain, based on a lookup keyed to the identifier, reference fingerprints representing the media stream without a need for the server to search for reference fingerprints that match query fingerprints representing the media stream being processed by the media client.

Assignments (3)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 058515/0809 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2021
From: CORMIE, STEVEN M.; JENTZ, BRIAN
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 058515/0845 →
Continuity (3)
Continuation 15929210 · Jan 28, 2020
Provisional Application 62900037 · Sep 13, 2019
Related Publication 20220109715A1 · Apr 7, 2022