IP Library Granted Patent US 9,706,261
Granted Patent B2
US 9,706,261 · App. 15/266,524 · Granted Jul 11, 2017

Detecting channel change in automatic content recognition fingerprint matching

Inventor: Juikun Chen (West Jordan, UT)
H04N21/812G06F17/30327H04N21/222H04N21/23424H04N21/2407H04N21/2668H04N21/4312H04N21/44008H04N21/44016H04N21/44222H04N21/4882H04N21/835H04N21/8352
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,706,261
App. No.
15/266,524
Granted
Jul 11, 2017
Kind
B2
Abstract

The disclosed system and method receives, from a media device, a query fingerprint that includes an ordered sequence of frames of content being consumed, and matches the query fingerprint with frame fingerprints by determining a distance between respective frame fingerprints and corresponding ones of the ordered sequence of frames. The matching may be performed by populating a binary tree structure with respective distances, chronologically across leaf nodes of the binary tree structure, and then noting a value of the root node. If below a maximum threshold, then there is a match. The system and method may detect a channel change at the media device by traversing a right or a left sub-tree of the binary tree structure, and locating a low value (or a zero) at the first or second nodes of the sub-tree. This channel change may be useful in targeting media to content being streamed to the media device.

Claims (42)

1. A system comprising:

Non-transitory computer-readable storage medium storing instructions and a database having a plurality of frame fingerprints associated with media programs being streamed to a plurality of channels; a processing device to execute the instructions to:

receive, from a media device, a query fingerprint of content being consumed, the query fingerprint including an ordered sequence of frames;

match the query fingerprint with the plurality of frame fingerprints stored in the database by determining a difference between respective frame fingerprints and corresponding ones of the ordered sequence of frames, wherein to match comprises to populate a binary tree structure with respective differences; and

detect a channel change of the media device from a first channel to a second channel responsive to locating a value, which is at or below a threshold maximum value, within a threshold number of children nodes from a root node upon traversing down the binary tree structure.

2. The system of claim 1 , wherein to detect the channel change, the processing device to locate the value at a first node of one of a right or a left sub-tree positioned below the root node.

3. The system of claim 1 , wherein to detect the channel change, the processing device to locate the value at a second node of a right or a left sub-tree positioned below the root node.

4. The system of claim 1 , wherein to populate the binary tree structure with respective differences, the processing device further to:

populate leaf nodes of the binary tree structure with values of the respective differences chronologically from one side to an opposite side of the binary tree structure; and

populate parent nodes with values derived from adding values of children nodes connected to the parent nodes.

5. The system of claim 1 , wherein the processing device further to determine an identity of the second channel by identifying a channel streaming content with frames matching those at an end of the ordered sequence of frames of the query fingerprint.

6. The system of claim 5 , wherein the processing device further to notify an advertisement (“ad”) targeter of the identity of the second channel being watched in response to detecting the channel change.

7. The system of claim 6 , further comprising the ad targeter, wherein the ad targeter is further to:

send a notification of the channel change and the identity of the second channel to an ad server located across a network; and

receive, from the ad server, a replacement ad or overlay content to target commercial slots of the second channel.

8. A method comprising:

receiving, from a media device, a query fingerprint of content being consumed, the query fingerprint including an ordered sequence of frames;

matching, by a processing device of a computing system, the query fingerprint with a plurality of frame fingerprints stored in a database in memory by determining a difference between respective frame fingerprints and corresponding ones of the ordered sequence of frames, wherein the matching comprises populating a binary tree structure with respective differences; and

detecting, by the processing device, a channel change of the media device from a first channel to a second channel responsive to locating a value, which is at or below a threshold maximum value, within a threshold number of children nodes from a root node upon traversing down the binary tree structure.

9. The method of claim 8 , wherein populating the binary tree structure with respective differences comprises:

populating leaf nodes of the binary tree structure with values of the respective differences chronologically from one side to an opposite side of the binary tree structure; and

populating parent nodes with values derived from adding values of children nodes connected to the parent nodes of the binary tree structure.

10. The method of claim 8 , further comprising determining an identity of the second channel by identifying a channel streaming content with frames matching those at an end of the ordered sequence of frames of the query fingerprint.

11. The method of claim 10 , further comprising:

notifying a content manager logic of the identity of the second channel being watched in response to detecting the channel change; and

targeting, by the content manager logic, advertising content streamed to the second channel.

12. The method of claim 8 , wherein the threshold maximum value is sixteen or less.

13. The method of claim 8 , wherein detecting the channel change comprises locating the value at a first node of one of a right or a left sub-tree positioned below the root node.

14. The method of claim 8 , wherein detecting the channel change comprises locating the value at a second node of one of a right or a left sub-tree positioned below the root node.

15. A non-transitory computer-readable storage medium storing instructions, wherein the instructions are executable by a processing device to:

receive, from a media device, a query fingerprint of content being consumed, the query fingerprint including an ordered sequence of frames;

match the query fingerprint with a plurality of frame fingerprints stored in a database in memory by determining a difference between respective frame fingerprints and corresponding ones of the ordered sequence of frames, wherein the matching comprises populating a binary tree structure with respective differences; and

detect a channel change of the media device from a first channel to a second channel responsive to locating a value, which is at or below a threshold maximum value, within a threshold number of children nodes from a root node upon traversing down the binary tree structure.

16. The non-transitory computer-readable storage medium of claim 15 , wherein to detect the channel change, the instructions further executable by the processing device to locate the value at one of a first node or a second node of one of a right or a left sub-tree positioned below the root node.

17. The non-transitory computer-readable storage medium of claim 15 , wherein to populate the binary tree structure with respective differences, the instructions are further executable by the processing device to:

populate leaf nodes of the binary tree structure with values of the respective differences chronologically from one side to an opposite side of the binary tree structure; and

populate parent nodes with values derived from adding values of children nodes connected to the parent nodes.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions are further executable by the processing device to determine an identity of the second channel by identifying a channel streaming content with frames matching those at an end of the ordered sequence of frames of the query fingerprint.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions are further executable by the processing device to notify a content manager logic of the identity of the second channel being watched in response to detecting the channel change.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the instructions are further executable by the content manager logic to:

send a notification of the channel change and the identity of the second channel to an ad server located across a network; and

receive, from the ad server, a replacement ad or overlay content to target commercial slots of the second channel.

Assignments (14)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2019
From: SORENSON MEDIA, INC.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 048355/0949 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2016
From: CHEN, JUIKUN
To: SORENSON MEDIA, INC.
Reel/Frame 039758/0597 →
Continuity (3)
Continuation 14815256 · Jul 31, 2015
Provisional Application 62175145 · Jun 12, 2015
Related Publication 20170013324A1 · Jan 12, 2017