IP Library Granted Patent US 10,685,221
Granted Patent B2
US 10,685,221 · App. 16/057,302 · Granted Jun 16, 2020

Methods and apparatus to monitor environments

Inventors: Morris Lee (Palm Harbor, FL); Alejandro Terrazas (Santa Cruz, CA)
Assignee: The Nielsen Company (US), LLC
G06K9/00362G01B11/254G06F3/0304G06K9/00771G06K9/2036G06T7/593H04H60/66H04N13/261H04N21/44218
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,685,221
App. No.
16/057,302
Granted
Jun 16, 2020
Kind
B2
Abstract

Methods and apparatus to monitor environments are disclosed. Example audience measurement devices disclosed herein include means for executing a first recognition analysis on three-dimensional data representative of a first object within a threshold distance from a three-dimensional sensor. Disclosed example audience measurement devices also include means for executing a second recognition analysis on two-dimensional data representative of a second object outside the threshold distance from the three-dimensional sensor. Disclosed example audience measurement devices further include means for combining a first detection of a first person provided by the first recognition analysis and a second detection of a second person provided by the second recognition analysis to generate a people count for an environment.

Claims (39)

1. An audience measurement device comprising:

means for executing a first recognition analysis on three-dimensional data representative of a first object within a threshold distance from a three-dimensional sensor;

means for executing a second recognition analysis on two-dimensional data representative of a second object outside the threshold distance from the three-dimensional sensor; and

means for combining a first detection of a first person provided by the first recognition analysis and a second detection of a second person provided by the second recognition analysis to generate a people count for an environment.

2. The audience measurement device as defined in claim 1 , further including means for detecting the first person travelling from within the threshold distance to outside the threshold distance, the means for detecting to record a location of the first person when the first person travelled outside the threshold distance.

3. The audience measurement device as defined in claim 2 , further including means for focusing the second recognition analysis on the location of the first person.

4. The audience measurement device as defined in claim 3 , wherein the means for focusing is to define a boundary within the two-dimensional data in which to perform the second recognition analysis, the boundary corresponding to areas adjacent to the location of the first person.

5. The audience measurement device as defined in claim 4 , wherein a shape of the boundary depends on a type of body part detected by the first recognition analysis.

6. An audience measurement device comprising:

means for analyzing a plurality of three-dimensional data points having respective depth values representative of distances from a sensor of an environment;

means for executing a three-dimensional recognition analysis on a first area of the environment corresponding to a first set of the three-dimensional data points when the three-dimensional data points in the first set of the three-dimensional data points have respective first depth values satisfying a threshold; and

means for executing a two-dimensional recognition analysis on a second area of the environment corresponding to a first set of two-dimensional data points representative of at least a portion of the environment corresponding to a second set of three-dimensional data points when the three-dimensional data points in the second set of the three-dimensional data points have respective second depth values not satisfying the threshold.

7. The audience measurement device as defined in claim 6 , further including:

means for capturing the three-dimensional data points using electromagnetic beams projected in a pattern; and

means for capturing the two-dimensional data points.

8. The audience measurement device as defined in claim 6 , wherein the means for executing the two-dimensional recognition analysis is to use a two-dimensional coordinate of the first set of the two-dimensional data points in a first frame to focus the two-dimensional recognition analysis on the first area of the environment for a second frame subsequent to the first frame.

9. The audience measurement device as defined in claim 6 , further including means for triggering the means for executing the two-dimensional recognition analysis to execute in response to elapsing of a time interval.

10. The audience measurement device as defined in claim 6 , wherein the means for executing the three-dimensional recognition analysis and the means for executing the two-dimensional recognition analysis are to determine whether an object of the environment corresponds to a person, and further including means for calculating a people count using results of the at least one of the three-dimensional recognition analysis or the two-dimensional recognition analyses.

11. An audience measurement device comprising:

means for storing a first frame of data and a second frame of data; and

means for analyzing data by at least:

triggering a two-dimensional recognition analysis, in connection with the first frame of data, on two-dimensional data points representative of an object detected in an environment, the triggering based on satisfying a first trigger event, the first trigger event being one of (1) a distance between the object and a sensor of the environment satisfying a threshold distance, or (2) elapsing of a time interval; and

triggering the two-dimensional recognition analysis in connection with the second frame in response to determining that the object in the first frame is recognized as a person, the second frame subsequent to the first frame, the two-dimensional recognition analysis of the second frame performed on two-dimensional data points representative of a location in the second frame corresponding to a location of the person in the first frame,

wherein the two-dimensional recognition analysis in connection with the second frame is limited to a portion of the second frame corresponding to a shape surrounding the location in the second frame.

12. The audience measurement device as defined in claim 11 , wherein the sensor is to capture three-dimensional data points using electromagnetic beams projected in a pattern, and further including means for capturing the two-dimensional data points.

13. The audience measurement device as defined in claim 12 , further including means for analyzing a plurality of the three-dimensional data points to determine the distance between the object and the sensor.

14. The audience measurement device as defined in claim 11 , further including means for calculating a people count using results of the two-dimensional recognition analysis.

15. An audience measurement device comprising:

means for storing a first frame of data and a second frame of data; and

means for analyzing data by at least:

triggering a two-dimensional recognition analysis, in connection with the first frame of data, on two-dimensional data points representative of an object detected in an environment, the triggering based on satisfying a first trigger event, the first trigger event being one of (1) a distance between the object and a sensor of the environment satisfying a threshold distance, or (2) elapsing of a time interval; and

triggering the two-dimensional recognition analysis in connection with the second frame in response to determining that the object in the first frame is recognized as a person, the second frame subsequent to the first frame, the two-dimensional recognition analysis of the second frame performed on two-dimensional data points representative of a location in the second frame corresponding to a location of the person in the first frame,

wherein the means for analyzing data is not to trigger the two-dimensional recognition analysis in connection with the second frame when the object in the first frame is not recognized as a person.

16. An audience measurement device comprising:

means for storing a first frame of data and a second frame of data; and

means for analyzing data by at least:

triggering a two-dimensional recognition analysis, in connection with the first frame of data, on two-dimensional data points representative of an object detected in an environment, the triggering based on satisfying a first trigger event, the first trigger event being one of (1) a distance between the object and a sensor of the environment satisfying a threshold distance, or (2) elapsing of a time interval; and

triggering the two-dimensional recognition analysis in connection with the second frame in response to determining that the object in the first frame is recognized as a person, the second frame subsequent to the first frame, the two-dimensional recognition analysis of the second frame performed on two-dimensional data points representative of a location in the second frame corresponding to a location of the person in the first frame; and

means for performing a three-dimensional recognition analysis on three-dimensional data points representative of the object detected in the environment when the first trigger event is not satisfied.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2019
From: LEE, MORRIS; TERRAZAS, ALEJANDRO
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 048322/0037 →