IP Library Granted Patent US 10,839,225
Granted Patent B2
US 10,839,225 · App. 16/033,018 · Granted Nov 17, 2020

Methods and apparatus to monitor a split screen media presentation

Inventors: Justin David Lee Fahnestock (Tampa, FL); John C. Kistenmacher (Brunswick, ME); Ronan Heffernan (Wesley Chapel, FL); Scott Daniel Barraclough (New Port Richey, FL); John Wesley Kercher, IV (Tampa, FL)
Assignee: The Nielsen Company (US), LLC
G06K9/00744G06K9/3233G06T7/11G06T7/13H04H60/58H04H60/59
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,839,225
App. No.
16/033,018
Granted
Nov 17, 2020
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture are disclosed to monitor a split screen media presentation. Example apparatus disclosed herein include an audio processor to process audio output from a media device to determine audio metering data to identify first media presented by the media device. Disclosed example apparatus also include a video signature generator to generate one or more video signatures associated with video output from the media device. Disclosed example apparatus further include a computer vision processor to identify a first region-of-interest of a split screen presentation of the media device including the first media in response to a determination that the one or more generated video signatures do not match reference video signatures corresponding to the first media.

Claims (52)

1. An apparatus, comprising:

an audio processor to process audio output from a media device to determine audio metering data to identify first media presented by the media device;

a video signature generator to generate one or more video signatures associated with video output from the media device; and

a computer vision processor to identify a first region-of-interest of a split screen presentation of the media device including the first media in response to a determination that the one or more generated video signatures do not match reference video signatures corresponding to the first media.

2. The apparatus of claim 1 , wherein the audio metering data includes at least one of audio signatures or audio watermarks.

3. The apparatus of claim 1 , further including a monitoring data communicator to:

transmit the audio metering data and the one or more generated video signatures to a central facility; and

receive, from the central facility, an indication that the one or more generated video signatures do not match reference video signatures.

4. The apparatus of claim 1 , further including an image creator to capture one or more images representative of the video output from the media device, the computer vision processor to process the one or more captured images to identify candidate regions-of-interest, and to cause the video signature generator to generate respective video signatures for respective ones of the candidate regions-of-interest, the first region-of-interest corresponding to a matching region-of-interest having video signatures corresponding to the first media.

5. The apparatus of claim 4 , wherein the computer vision processor is to determine that the first region-of-interest includes the first media in response to receiving an indication from a central facility that the respective video signatures for the first region-of-interest correspond to the first media.

6. The apparatus of claim 4 , wherein the computer vision processor is to identify the candidate regions-of-interest using at least one process of edge detection, line detection, feature detection, or feature extraction.

7. The apparatus of claim 4 , wherein the computer vision processor includes:

an image variation detector to:

divide the one or more captured images representative of the video output of the media device into a plurality of sub-regions; and

determine inter-frame variation associated with each sub-region;

an image sub-region encoder to create one or more encoded synthetic images representative of the inter-frame variation associated with each sub-region; and

a region-of-interest segmenter to segment the one or more synthetic images into candidate regions-of-interest.

8. A method to monitor a split screen presentation of a media device, the method comprising:

processing, by executing an instruction with a processor, audio output from a media device to determine audio metering data to identify first media presented by the media device;

generating, by executing an instruction with the processor, one or more video signatures associated with video output from the media device; and

identifying, by executing an instruction with the processor, a first region-of-interest of a split screen presentation of the media device including the first media in response to determining that the one or more generated video signatures do not match reference video signatures corresponding to the first media.

9. The method of claim 8 , wherein the audio metering data includes at least one of audio signatures or audio watermarks.

10. The method of claim 8 , further including:

transmitting audio metering data and the one or more generated video signatures to a central facility; and

receiving, from the central facility, an indication that the one or more generated video signatures do not match reference video signatures.

11. The method of claim 8 , further including:

capturing one or more images representative of the video output from the media device;

processing the one or more captured images to identify candidate regions-of-interest, and

generating video signatures for respective ones of the candidate regions-of-interest, the first region-of-interest corresponding to a matching region-of-interest having video signatures corresponding to the first media.

12. The method of claim 11 , wherein the identifying of the first region-of-interest includes determining the first region-of-interest includes the first media in response to receiving an indication from a central facility that the respective video signatures for a first candidate region-of-interest of the candidate regions-of-interest correspond to the first media.

13. The method of claim 11 , wherein the identifying of the candidate regions-of-interest is implemented by at least one process of edge detection, line detection, feature detection, or feature extraction.

14. The method of claim 11 , further including:

dividing the one or more captured images representative of the video output of the media device into a plurality of sub-regions;

determining inter-frame variation associated with each sub-region;

creating one or more encoded synthetic images representative of the inter-frame variation associated with each sub-region; and

segmenting the one or more synthetic images into candidate regions-of-interest.

15. A non-transitory computer readable storage medium comprising instructions that, when executed, cause a machine to, at least:

process audio output from a media device to determine audio metering data to identify first media presented by the media device;

generate one or more video signatures associated with video output from the media device; and

identify a first region-of-interest of a split screen presentation of the media device including the first media in response to a determination that the one or more generated video signatures do not match reference video signatures corresponding to the first media.

16. The non-transitory computer readable storage medium as defined in claim 15 , wherein the audio metering data includes at least one of audio signatures or audio watermarks.

17. The non-transitory computer readable storage medium as defined in claim 15 , further including instructions that, when executed, cause the machine to transmit audio metering data and the one or more generated video signatures to a central facility and receive, from the central facility, an indication that the one or more generated video signatures do not match reference video signatures.

18. The non-transitory computer readable storage medium as defined in claim 15 , further including instructions that, when executed, cause the machine to:

capture one or more images representative of the video output from the media device;

process the one or more captured images to identify candidate regions-of-interest; and

generate video signatures for respective ones of the candidate regions-of-interest, the first region-of-interest corresponding to a matching region-of-interest having video signatures corresponding to the first media.

19. The non-transitory computer readable storage medium of claim 18 , wherein the identifying of the first region-of-interest includes determining the first region-of-interest includes the first media in response to receiving an indication from a central facility that the respective video signatures for a first candidate region-of-interest of the candidate regions-of-interest correspond to the first media.

20. The non-transitory computer readable storage medium as defined in claim 18 , further including instructions that, when executed, cause the machine to:

divide the one or more captured images representative of the video output of the media device into a plurality of sub-regions; and

determine inter-frame variation associated with each sub-region;

create one or more encoded synthetic images representative of the inter-frame variation associated with each sub-region; and

segment the one or more synthetic images into candidate regions-of-interest.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 6, 2020
From: KISTENMACHER, JOHN C.; HEFFERNAN, RONAN; FAHNESTOCK, JUSTIN DAVID LEE; BARRACLOUGH, SCOTT DANIEL; KERCHER, JOHN WESLEY, IV
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 053988/0750 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →