IP Library Granted Patent US 10,631,049
Granted Patent B2
US 10,631,049 · App. 16/274,966 · Granted Apr 21, 2020

Media channel identification with video multi-match detection and disambiguation based on audio fingerprint

Inventors: Chung Won Seo (Seoul, KR); Youngmoo Kwon (Goyang, KR); Jaehyung Lee (Seoul, KR)
Assignee: Gracenote, Inc.
H04N21/44204G06F16/683G06K9/00744G06K9/00758H04H60/43H04H60/58H04H60/59H04N21/23418H04N21/435H04N21/4394H04N21/44008H04N21/4622H04N21/6118H04N21/812H04N21/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,631,049
App. No.
16/274,966
Granted
Apr 21, 2020
Kind
B2
Abstract

Disclosed are methods and systems to help disambiguate channel identification in a scenario where a video fingerprint of media content matches multiple reference video fingerprints corresponding respectively with multiple different channels. Given such a multi-match situation, an entity could disambiguate based on an audio component of the media content, such as by further determining that an audio fingerprint of the media content at issue matches an audio fingerprint of just one of the multiple channels, thereby establishing that that is the channel on which the media content being rendered by the media presentation device is arriving.

Claims (46)

1. A media presentation device comprising:

a media input interface through which to receive media content to be presented by the media presentation device;

a media presentation interface for presenting the media content;

a network communication interface;

a processing unit;

non-transitory data storage; and

program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations including:

generating digital video fingerprint data representing the media content, based on analysis of the media content, and outputting the generated digital video fingerprint data for transmission through the network communication interface to a server,

after outputting the digital video fingerprint data for transmission to the server, receiving from the server a request for audio fingerprint data representing the media content, the request being responsive to the server determining that the digital video fingerprint data matches reference video fingerprint data corresponding with multiple channels,

responsive to the request, outputting digital audio fingerprint data representing the media content, for transmission of the digital audio fingerprint data through the network communication interface to the server, and

causing the media presentation device to present, in conjunction with the media content, supplemental channel-specific content associated with one of the multiple channels, the one channel being identified from among the multiple channels based on a determination that the digital audio fingerprint data matches reference audio fingerprint data representing just the one channel of the multiple channels.

2. The media presentation device of claim 1 , wherein the media content has a video track and an audio track, wherein the digital video fingerprint data is fingerprint data representing the video track and the digital audio fingerprint data fingerprint data representing the audio track.

3. The media presentation device of claim 1 , wherein the digital audio fingerprint data represents at least a language track of the media content.

4. The media presentation device of claim 1 , wherein the operations further include:

responsive to the request, generating the digital audio fingerprint data representing the media content.

5. The media presentation device of claim 1 , wherein the supplemental channel-specific content comprises at least one of a pop-up advertisement, a commercial break, or a channel-identification.

6. The media presentation device of claim 1 ,

wherein the supplemental channel-specific content is an advertisement, and

wherein causing the media presentation device to present the supplemental content in conjunction with the media content comprises causing the media presentation device to present the advertisement in place of a portion of the media content.

7. The media presentation device of claim 1 , wherein the media presentation device comprises a television.

8. A method comprising:

generating by a media presentation device, based on analysis of media content being rendered by the media presentation device, digital video fingerprint data representing the media content being rendered by the media presentation device;

outputting by the media presentation device the generated video fingerprint data for transmission over a network to a server;

after outputting the digital video fingerprint data for transmission over the network to the server, receiving by the media presentation device, from the server, a request for audio fingerprint data representing the media content, the request being responsive to the server determining that the digital video fingerprint data matches reference video fingerprint data corresponding with multiple channels;

responsive to the request, outputting by the media presentation device digital audio fingerprint data representing the media content, for transmission of the digital audio fingerprint data over the network to the server; and

presenting, by the media presentation device, in conjunction with the media content, supplemental channel-specific content associated with one of the multiple channels, the one channel being identified from among the multiple channels based on a determination that the digital audio fingerprint data matches reference audio fingerprint data of just the one channel of the multiple channels.

9. The method of claim 8 , wherein the media content has a video track and an audio track, wherein the digital video fingerprint data is fingerprint data representing the video track and the digital audio fingerprint data fingerprint data representing the audio track.

10. The method of claim 8 , wherein the digital audio fingerprint data represents at least a language track of the media content.

11. The method of claim 8 , further comprising:

responsive to the request, generating the digital audio fingerprint data representing the media content.

12. The method of claim 8 , wherein the supplemental channel-specific content comprises at least one of a pop-up advertisement, a commercial break, or a channel-identification.

13. The method of claim 8 ,

wherein the supplemental channel-specific content is an advertisement, and

wherein presenting the supplemental content in conjunction with the media content comprises presenting the advertisement in place of a portion of the media content.

14. A method comprising:

determining, by a computing system, that digital video fingerprint data representing media content being rendered by a media presentation device matches reference video fingerprint data corresponding with multiple channels;

responsive to at least determining that the digital video fingerprint data matches the reference video fingerprint data corresponding with multiple channels, performing disambiguation by the computing system based at least in part on a determination that digital audio fingerprint data representing the media content being rendered by the media presentation device matches reference audio fingerprint data corresponding with just a single channel of the multiple channels, the disambiguation establishing that the media presentation device is receiving the media content on the single channel; and

taking action based on the establishing that the media presentation device is receiving the media content on the single channel.

15. The method of claim 14 , wherein the media content has a video track and an audio track, wherein the digital video fingerprint data is fingerprint data representing the video track and the digital audio fingerprint data fingerprint data representing the audio track.

16. The method of claim 14 , wherein the digital audio fingerprint data represents at least a language track of the media content.

17. The method of claim 14 , wherein taking action based on the establishing that the media presentation device is receiving the media content on the single channel comprises causing the media presentation device to present supplemental channel-specific content in conjunction with the media content being rendered by the media presentation device.

18. The method of claim 17 , wherein the supplemental channel-specific content includes at least one of a pop-up advertisement, a commercial break, or a channel-identification.

19. The method of claim 17 ,

wherein the supplemental channel-specific content comprises an advertisement, and

wherein causing the media presentation device to present the supplemental channel-specific content in conjunction with the media content being rendered by the media presentation device comprises causing the media presentation device to present the advertisement as a replacement for a portion of the media content.

20. The method of claim 14 , wherein taking action based on the establishing that the media presentation device is receiving the media content on the single channel comprises inserting an advertisement in place of a portion of the media content.

Assignments (10)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: GRACENOTE, INC.
To: ROKU, INC.
Reel/Frame 056103/0786 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2019
From: SEO, CHUNG WON; KWON, YOUNGMOO; LEE, JAEHYUNG
To: GRACENOTE, INC.
Reel/Frame 048384/0960 →