IP Library Granted Patent US 9,516,377
Granted Patent B1
US 9,516,377 · App. 14/815,256 · Granted Dec 6, 2016

Detecting channel change in automatic content recognition fingerprint matching

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,516,377
App. No.
14/815,256
Granted
Dec 6, 2016
Kind
B1
Abstract

The disclosed system and method receives, from a media device, a query fingerprint that includes an ordered sequence of frames of content being consumed, and matches the query fingerprint with frame fingerprints by determining a distance between respective frame fingerprints and corresponding ones of the ordered sequence of frames. The matching may be performed by populating a binary tree structure with respective distances, chronologically across leaf nodes of the binary tree structure, and then noting a value of the root node. If below a maximum threshold, then there is a match. The system and method may detect a channel change at the media device by traversing a right or a left sub-tree of the binary tree structure, and locating a low value (or a zero) at the first or second nodes of the sub-tree. This channel change may be useful in targeting media to content being streamed to the media device.

Claims (53)

1. A system comprising:

computer-readable storage medium storing instructions and a database having a plurality of frame fingerprints associated with media programs being streamed to a plurality of channels;

at least one processing device to execute the instructions to:

receive, from a media device, a query fingerprint of content being consumed, the query fingerprint including an ordered sequence of frames;

match the query fingerprint with the plurality of frame fingerprints stored in the database by determining a distance between respective frame fingerprints and corresponding ones of the ordered sequence of frames, wherein to match comprises to populate a binary tree structure with respective distances;

detect a channel change from a first channel to a second channel responsive to locating a zero value within a threshold number of children nodes from a root node when traversing down the binary tree structure; and

determine an identity of the second channel by identifying a channel streaming content with frames matching those at an end of the ordered sequence of frames of the query fingerprint.

2. The system of claim 1 , wherein to detect the channel change, the processing device to locate the zero value at a first node of a right or a left sub-tree positioned below the root node.

3. The system of claim 1 , wherein to detect the channel change, the processing device to locate the zero value at a second node of a right or a left sub-tree positioned below the root node.

4. The system of claim 1 , wherein to populate the binary tree structure with respective distances, the at least one processing device further to:

populate leaf nodes of the binary tree structure with values of the respective distances chronologically from one side to an opposite side; and

populate parent nodes with values derived from adding values of children nodes connected to the parent nodes.

5. The system of claim 1 , where the at least one processing device further to notify an advertisement (“ad”) targeter of the identity of the second channel being watched in response to detecting the channel change so that the ad targeter can adjust targeting media to content streamed to the second channel.

6. The system of claim 5 , wherein to adjust targeting media to the content streamed to the second channel comprises to update an overlay or replacement advertisement to, respectively, overlay on top of or to replace an advertisement streamed to the second channel.

7. The system of claim 5 , wherein the ad targeter is further to:

send a notification of the channel change and the identity of the second channel to an ad server located across a network; and

receive, from the ad server, a replacement ad or overlay content to target commercial slots of the second channel.

8. A method comprising:

receiving, from a media device, a query fingerprint of content being consumed, the query fingerprint including an ordered sequence of frames;

matching, using at least one processing device, the query fingerprint with a plurality of frame fingerprints stored in a database in memory by determining a distance between respective frame fingerprints and corresponding ones of the ordered sequence of frames, to identify a matching fingerprint, wherein the matching comprises:

populating leaf nodes of a binary tree structure with values of respective distances chronologically from one side to an opposite side of the binary tree structure;

populating parent nodes with values derived from adding values of children nodes connected to the parent nodes of the binary tree structure; and

detecting a value of or lower than a preset maximum threshold at a root node of the binary tree structure as indicating a match; and

notifying content manager logic of a channel corresponding to the matching fingerprint so that the content manager logic can target media to content streamed to the channel.

9. The method of claim 8 , wherein the channel is a first channel, the method further comprising:

detecting a channel change from the first channel to a second channel responsive to locating a low value within a threshold number of children nodes from the root node when traversing down the binary tree structure, wherein the low value is at or below the preset maximum threshold; and

notifying the content manager logic of the channel change to the second channel.

10. The method of claim 9 , wherein the preset maximum threshold is zero.

11. The method of claim 9 , wherein detecting the channel change comprises locating the low value at a first node of a right or a left sub-tree positioned below the root node.

12. The method of claim 9 , wherein detecting the channel change comprises locating the low value at a second node of a right or a left sub-tree positioned below the root node.

13. The method of claim 9 , further comprising:

sending a notification of the second channel to an ad server located across a network; and

receiving, from the ad server, a replacement ad or overlay content to target commercial slots of the second channel.

14. A server comprising:

computer-readable storage medium storing instructions and a database having a plurality of frame fingerprints associated with media programs being streamed to a plurality of channels;

at least one processing device to execute the instructions to:

receive, from a media device, a query fingerprint of content being consumed, the query fingerprint including an ordered sequence of frames;

match the query fingerprint with a plurality of frame fingerprints stored in the database by determining a distance between respective frame fingerprints and corresponding ones of the ordered sequence of frames, to identify a matching fingerprint, wherein to match comprises to:

populate leaf nodes of a binary tree structure with values of respective distances chronologically from one side to an opposite side of the binary tree structure;

populate parent nodes with values derived from adding values of children nodes connected to the parent nodes of the binary tree structure; and

detect a value of or lower than a preset maximum threshold at a root node of the binary tree structure as indicating a match; and

notify content manager logic of a channel corresponding to the matching fingerprint so that the content manager logic can target media to content streamed to the channel.

15. The server of claim 14 , wherein the channel is a first channel, the at least one processing device further to:

detect a channel change from the first channel to a second channel responsive to locating a low value within a threshold number of children nodes from the root node when traversing down the binary tree structure, wherein the low value is at or below the preset maximum threshold;

determine an identity of the second channel by identifying a channel streaming content with frames matching those at an end of the ordered sequence of frames of the query fingerprint; and

notify the content manager logic of the channel change and the identity of the second channel.

16. The server of claim 15 , wherein the preset maximum threshold is zero.

17. The server of claim 16 , wherein to detect the channel change comprises to locate the zero value at a first node of a right or a left sub-tree positioned below the root node.

18. The server of claim 16 , wherein to detect the channel change comprises to locate the zero value at a second node of a right or a left sub-tree positioned below the root node.

19. The server of claim 15 , wherein the at least one processing device further to:

send a notification of the second channel to an ad server located across a network; and

receive, from the ad server, a replacement ad or overlay content to target commercial slots of the second channel.

20. The method of claim 9 further comprising determining an identity of the second channel by identifying a channel streaming content with frames matching those at an end of the ordered sequence of frames of the query fingerprint.

Assignments (11)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2019
From: SORENSON MEDIA, INC.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 048355/0949 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 31, 2015
From: CHEN, JUIKUN
To: SORENSON MEDIA, INC.
Reel/Frame 036232/0657 →