IP Library Granted Patent US 10,149,007
Granted Patent B2
US 10,149,007 · App. 15/842,189 · Granted Dec 4, 2018

Media channel identification with video multi-match detection and disambiguation based on audio fingerprint

Inventors: Chung Won Seo (Seoul, KR); Youngmoo Kwon (Goyang, KR); Jaehyung Lee (Seoul, KR)
Assignee: Gracenote, Inc.
H04N21/44204H04N21/435H04N21/4394H04N21/44008H04N21/4622H04N21/6118H04N21/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,149,007
App. No.
15/842,189
Granted
Dec 4, 2018
Kind
B2
Abstract

Disclosed are methods and systems to help disambiguate channel identification in a scenario where a video fingerprint of media content matches multiple reference video fingerprints corresponding respectively with multiple different channels. Given such a multi-match situation, an entity could disambiguate based on an audio component of the media content, such as by further determining that an audio fingerprint of the media content at issue matches an audio fingerprint of just one of the multiple channels, thereby establishing that that is the channel on which the media content being rendered by the media presentation device is arriving.

Claims (47)

1. A method comprising:

determining, by a computing system, that a digital video fingerprint of media content being rendered by a media presentation device matches multiple reference video fingerprints each corresponding with a different respective channel;

responsive to at least determining that the digital video fingerprint matches the multiple reference video fingerprints each corresponding with a different respective channel, performing disambiguation by the computing system based at least in part on a determination that a digital audio fingerprint of the media content being rendered by the media presentation device matches a reference audio fingerprint corresponding with just a single channel of the channels with which the multiple reference video fingerprints correspond, the disambiguation establishing that the media presentation device is receiving the media content on the single channel; and

taking action by the computing system based on the establishing that the media presentation device is receiving the media content on the single channel.

2. The method of claim 1 , wherein the media content being rendered by the media presentation device has a video track and an audio track, wherein the digital video fingerprint is a fingerprint of the video track and the digital audio fingerprint is a fingerprint of the audio track.

3. The method of claim 1 , wherein the digital audio fingerprint of the media content being rendered by the media presentation device represents at least a language track of the media content being rendered by the media presentation device.

4. The method of claim 1 , wherein the computing system is an entity other than the media presentation device, and wherein the digital video fingerprint and digital audio fingerprint are generated by the media presentation device, the method further comprising:

receiving by the computing system, from the media presentation device, the digital video fingerprint and the digital audio fingerprint.

5. The method of claim 1 , wherein the multiple reference video fingerprints are selected from a plurality of reference video fingerprints in reference data, and wherein determining that the digital video fingerprint of the media content being rendered by the media presentation device matches the multiple reference video fingerprints comprises:

comparing the digital video fingerprint of the media content being rendered by the media presentation device with the plurality of reference video fingerprints in the reference data, and

based on the comparing, determining that the digital video fingerprint of the media content being rendered by the media presentation device matches the multiple reference video fingerprints.

6. The method of claim 1 , further comprising detecting and flagging a multi-match group consisting of the multiple reference video fingerprints,

wherein determining that the digital video fingerprint of the media content being rendered by the media presentation device matches the multiple reference video fingerprints comprises determining that the digital video fingerprint of the media content being rendered by the media presentation device matches the reference video fingerprints of the flagged multi-match group.

7. The method of claim 6 , further comprising:

comparing reference audio fingerprints corresponding with the reference video fingerprints of the flagged multi-match group;

detecting, based on the comparing, a difference between the compared reference audio fingerprints; and

responsive to detecting the difference, further flagging the multi-match group to indicate that audio-fingerprint analysis could facilitate disambiguation,

wherein performing disambiguation based on the determination that the digital audio fingerprint of the media content being rendered by the media presentation device matches a reference audio fingerprint corresponding with just a single channel is further responsive to determining that the multi-match group is further flagged to indicate that audio-fingerprint analysis could facilitate disambiguation.

8. The method of claim 1 , wherein the computing system is an entity other than the media presentation device, the method further comprising:

responsive to at least determining that the digital video fingerprint matches the multiple reference video fingerprints each corresponding with a different respective channel, requesting and receiving by the computing system from the media presentation device the digital audio fingerprint of the media content being rendered by the media presentation device, to facilitate the performing of disambiguation based on the obtained digital audio fingerprint.

9. The method of claim 1 , further comprising:

responsive to at least determining that the digital video fingerprint matches the multiple reference fingerprints each corresponding with a different channel, generating reference audio fingerprints corresponding with the channels with which the multiple reference video fingerprints correspond, to facilitate the disambiguation.

10. The method of claim 1 , wherein taking action based on the establishing that the media presentation device is receiving the media content on the single channel comprises causing supplemental channel-specific content to be presented in conjunction with the media content being rendered by the media presentation device.

11. The method of claim 1 , carried out at least in part by the media presentation device.

12. The method of claim 1 , wherein taking action based on the establishing that the media presentation device is receiving the media content on the single channel comprises recording presentation of the single channel for use in a channel ratings system.

13. A non-transitory computer readable medium having stored thereon instructions executable by a processing unit to carry out operations comprising:

determining that a digital video fingerprint of media content being rendered by a media presentation device matches multiple reference video fingerprints each corresponding with a different respective channel;

responsive to at least determining that the digital video fingerprint matches the multiple reference video fingerprints each corresponding with a different respective channel, performing disambiguation based at least in part on a determination that a digital audio fingerprint of the media content being rendered by the media presentation device matches a reference audio fingerprint corresponding with just a single channel of the channels with which the multiple reference fingerprints correspond, the disambiguation establishing that the media presentation device is receiving the media content on the single channel; and

taking action based on the establishing that the media presentation device is receiving the media content on the single channel.

14. The non-transitory computer readable medium of claim 13 , wherein the digital audio fingerprint of the media content being rendered by the media presentation device represents at least a language track of the media content being rendered by the media presentation device.

15. The non-transitory computer readable medium of claim 13 , wherein taking action based on the establishing that the media presentation device is receiving the media content on the single channel comprises causing supplemental channel-specific content to be presented in conjunction with the media content being rendered by the media presentation device.

16. The non-transitory computer readable medium of claim 13 , wherein taking action based on the establishing that the media presentation device is receiving the media content on the single channel comprises recording presentation of the single channel for use in a channel ratings system.

17. A system comprising:

a network communication interface;

a processing unit;

non-transitory data storage; and

program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations including:

receiving from a media presentation device, via the network communication interface, a digital video fingerprint of media content being rendered by the media presentation device,

determining that the received digital video fingerprint matches reference video fingerprints corresponding with multiple channels,

receiving from the media presentation device, via the network communication interface, a digital audio fingerprint of the media content being rendered by the media presentation device,

using the received digital audio fingerprint to determine which of the multiple channels carries the media content being rendered by the media presentation device, wherein using the received digital audio fingerprint to determine which of the multiple channels carries the media content being rendered by the media presentation device comprises determining that the received digital audio fingerprint matches a reference audio fingerprint corresponding with just a single channel of the multiple channels, and

taking action based on the determined channel.

18. The system of claim 17 , wherein the operations further comprise:

responsive to at least determining that the received matches the reference video fingerprints corresponding with the multiple channels, sending to the media presentation device a request for the digital audio fingerprint of the media content being rendered by the media presentation device,

wherein receiving from the media presentation device the digital audio fingerprint of the media content being rendered by the media presentation device occurs in response to sending the request.

19. The system of claim 17 , wherein taking action based on the determined channel comprises causing supplemental channel-specific content to be presented in conjunction with the media content being rendered by the media presentation device.

20. The system of claim 17 , wherein taking action based on the determined channel comprises recording presentation of the determined channel for use in a channel ratings system.

Assignments (10)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: GRACENOTE, INC.
To: ROKU, INC.
Reel/Frame 056103/0786 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2018
From: SEO, CHUNG WON; KWON, YOUNGMOO; LEE, JAEHYUNG
To: GRACENOTE, INC.
Reel/Frame 044573/0871 →
Continuity (5)
Continuation 15253354 · Aug 31, 2016
Continuation In Part 15222405 · Jul 28, 2016
Continuation In Part 15179143 · Jun 10, 2016
Provisional Application 62301616 · Feb 29, 2016
Related Publication 20180109841A1 · Apr 19, 2018