IP Library Granted Patent US 10,506,285
Granted Patent B2
US 10,506,285 · App. 15/990,367 · Granted Dec 10, 2019

Method and apparatus to count people

Inventors: Padmanabhan Soundararajan (Tampa, FL); Marko Usaj (Izola, SI); Venugopal Srinivasan (Tarpon Springs, FL)
Assignee: The Nielsen Company (US), LLC
H04N21/44218G06F16/5838G06K9/00228G06K9/00778H04H60/45H04N3/09H04N21/25891H04N21/4223H04N21/4334H04N21/44008H04N21/44222H04N21/6582G01B3/006G01B3/20G01B5/30G01C11/025G02B26/10G02B27/0093G04F3/06G06K7/06G06K9/00295G06K9/00832G06T7/11G06T2207/30201G09B5/02H04H60/33H04N5/33H04N5/76H04N5/91H04N5/93H04N7/125H04N7/188H04N9/24H04N9/87H04N17/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,506,285
App. No.
15/990,367
Granted
Dec 10, 2019
Kind
B2
Abstract

Methods and apparatus to count people are disclosed. Example people counting apparatus disclosed herein include a difference calculator to calculate a degree of similarity between a first characteristic dataset and a second characteristic dataset representative of face detections in images. Disclosed example people counting apparatus also include a limiter to store the first characteristic dataset and the second characteristic dataset in a plurality of characteristic datasets associated when the degree of similarity does not satisfy a threshold, and to store the first characteristic dataset in the plurality of characteristic datasets and discard the second characteristic dataset when the degree of similarity satisfies the threshold to limit a number of stored characteristic datasets. Disclosed example people counting apparatus further include a comparator to compare the plurality of characteristic datasets to each other to determine a number of unique faces in an environment during a first period of time.

Claims (43)

1. A people counting apparatus comprising:

a difference calculator to calculate a degree of similarity between a first characteristic dataset representative of a first face detection in a first image at a first image location and a second characteristic dataset representative of a second face detection in a second image at the first image location, the first and second images included in a plurality of images representative of an environment during a first period of time of a media presentation;

a limiter to:

store the first characteristic dataset and the second characteristic dataset in a plurality of characteristic datasets associated with the first period of time when the degree of similarity does not satisfy a threshold; and

store the first characteristic dataset in the plurality of characteristic datasets and discard the second characteristic dataset when the degree of similarity satisfies the threshold to limit a number of stored characteristic datasets associated with the first image location; and

a comparator to compare the plurality of characteristic datasets to each other to determine a number of unique faces in the environment during the first period of time, at least one of the difference calculator, the limiter, or the comparator implemented by hardware or at least one processor.

2. The apparatus of claim 1 , wherein the difference calculator is to calculate a distance between a first facial feature vector of the first characteristic dataset and a second facial feature vector of the second characteristic dataset to determine the degree of similarity between the first characteristic dataset and the second characteristic dataset.

3. The apparatus of claim 2 , wherein the first characteristic dataset includes a first set of facial feature vectors including the first facial feature vector, and the second characteristic dataset includes a second set of facial feature vectors including the second facial feature vector.

4. The apparatus of claim 3 , wherein the first facial feature vector includes a distance and a direction for a space between two facial features associated with the first face detection, and the second facial feature vector includes a distance and a direction for a space between two facial features associated with the second face detection.

5. The apparatus of claim 1 , wherein the limiter includes a setting to specify a maximum value for the number of stored characteristic datasets associated with the first image location.

6. The apparatus of claim 5 , wherein the degree of similarity is a first degree of similarity, and when the first characteristic dataset and the second characteristic dataset are stored in the plurality of characteristic datasets, the difference calculator is further to:

calculate a second degree of similarity between the first characteristic dataset and a third characteristic dataset representative of a third face detection in a third image at the first location; and

calculate a third degree of similarity between the second characteristic dataset and the third characteristic dataset.

7. The apparatus of claim 6 , wherein the limiter is further to:

store the first characteristic dataset, the second characteristic dataset and the third characteristic dataset in the plurality of characteristic datasets when the first degree of similarity, the second degree of similarity and the third degree of similarity do not satisfy the threshold and the number of stored characteristic datasets associated with the first image location does not exceed the maximum value; and

store two of the least similar of first characteristic dataset, the second characteristic dataset and the third characteristic dataset and discard the other of the first characteristic dataset, the second characteristic dataset and the third characteristic dataset when the first degree of similarity, the second degree of similarity and the third degree of similarity do not satisfy the threshold and the number of stored characteristic datasets associated with the first image location exceeds the maximum value.

8. A people counting method comprising:

calculating, by executing an instruction with a processor, a degree of similarity between a first characteristic dataset representative of a first face detection in a first image at a first image location and a second characteristic dataset representative of a second face detection in a second image at the first image location, the first and second images included in a plurality of images representative of an environment during a first period of time of a media presentation;

storing, by executing an instruction with the processor, the first characteristic dataset and the second characteristic dataset in a plurality of characteristic datasets associated with the first period of time in response to the degree of similarity not satisfying a threshold;

storing, by executing an instruction with the processor, the first characteristic dataset in the plurality of characteristic datasets and discarding, by executing an instruction with the processor, the second characteristic dataset in response to the degree of similarity satisfying a threshold to limit a number of stored characteristic datasets associated with the first image location; and

comparing, by executing an instruction with the processor, the plurality of characteristic datasets to each other to determine a number of unique faces in the environment during the first period of time.

9. The method of claim 8 , further including calculating a distance between a first facial feature vector of the first characteristic dataset and a second facial feature vector of the second characteristic dataset to determine the degree of similarity between the first characteristic dataset and the second characteristic dataset.

10. The method of claim 9 , wherein the first characteristic dataset includes a first set of facial feature vectors including the first facial feature vector, and the second characteristic dataset includes a second set of facial feature vectors including the second facial feature vector.

11. The method of claim 10 , wherein the first facial feature vector includes a distance and a direction for a space between two facial features associated with the first face detection, and the second facial feature vector includes a distance and a direction for a space between two facial features associated with the second face detection.

12. The method of claim 8 , further including specifying a maximum value for the number of stored characteristic datasets associated with the first image location.

13. The method of claim 12 , wherein the degree of similarity is a first degree of similarity, and further including, when the first characteristic dataset and the second characteristic dataset are stored in the plurality of characteristic datasets, calculating a second degree of similarity between the first characteristic dataset and a third characteristic dataset representative of a third face detection in a third image at the first location, and calculating a third degree of similarity between the second characteristic dataset and the third characteristic dataset.

14. The method of claim 13 , further including:

storing the first characteristic dataset, the second characteristic dataset and the third characteristic dataset in the plurality of characteristic datasets when the first degree of similarity, the second degree of similarity and the third degree of similarity do not satisfy the threshold and the number of stored characteristic datasets associated with the first image location does not exceed the maximum value; and

storing two of the least similar of first characteristic dataset, the second characteristic dataset and the third characteristic dataset and discarding the other of the first characteristic dataset, the second characteristic dataset and the third characteristic dataset when the first degree of similarity, the second degree of similarity and the third degree of similarity do not satisfy the threshold and the number of stored characteristic datasets associated with the first image location exceeds the maximum value.

15. A non-transitory computer readable medium comprising instruction that, when executed, cause a machine to least:

calculate a degree of similarity between a first characteristic dataset representative of a first face detection in a first image at a first image location and a second characteristic dataset representative of a second face detection in a second image at the first image location, the first and second images included in a plurality of images representative of an environment during a first period of time of a media presentation;

store the first characteristic dataset and the second characteristic dataset in a plurality of characteristic datasets associated with the first period of time when the degree of similarity does not satisfy a threshold;

store the first characteristic dataset in the plurality of characteristic datasets and discard the second characteristic dataset when the degree of similarity satisfies the threshold to limit a number of stored characteristic datasets associated with the first image location; and

compare the plurality of characteristic datasets to each other to determine a number of unique faces in the environment during the first period of time.

16. The non-transitory computer readable medium as defined in claim 15 , wherein the instructions, when executed, cause the machine to calculate a distance between a first facial feature vector of the first characteristic dataset and a second facial feature vector of the second characteristic dataset to determine the degree of similarity between the first characteristic dataset and the second characteristic dataset.

17. The non-transitory computer readable medium as defined in claim 16 , wherein the first characteristic dataset includes a first set of facial feature vectors including the first facial feature vector, and the second characteristic dataset includes a second set of facial feature vectors including the second facial feature vector.

18. The non-transitory computer readable medium as defined in claim 17 , wherein the first facial feature vector includes a distance and a direction for a space between two facial features associated with the first face detection, and the second facial feature vector includes a distance and a direction for a space between two facial features associated with the second face detection.

19. The non-transitory computer readable medium as defined in claim 15 , wherein the instructions, when executed, cause the machine to specify a maximum value for the number of stored characteristic datasets associated with the first image location.

20. The non-transitory computer readable medium as defined in claim 19 , wherein the degree of similarity is a first degree of similarity, and when the first characteristic dataset and the second characteristic dataset are stored in the plurality of characteristic datasets, the instructions, when executed, further cause the machine to:

calculate a second degree of similarity between the first characteristic dataset and a third characteristic dataset representative of a third face detection in a third image at the first location;

calculate a third degree of similarity between the second characteristic dataset and the third characteristic dataset;

store the first characteristic dataset, the second characteristic dataset and the third characteristic dataset in the plurality of characteristic datasets when the first degree of similarity, the second degree of similarity and the third degree of similarity do not satisfy the threshold and the number of stored characteristic datasets associated with the first image location does not exceed the maximum value; and

store two of the least similar of first characteristic dataset, the second characteristic dataset and the third characteristic dataset and discard the other of the first characteristic dataset, the second characteristic dataset and the third characteristic dataset when the first degree of similarity, the second degree of similarity and the third degree of similarity do not satisfy the threshold and the number of stored characteristic datasets associated with the first image location exceeds the maximum value.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2018
From: SOUNDARARAJAN, PADMANABHAN; USAJ, MARKO; SRINIVASAN, VENUGOPAL
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 047346/0172 →