IP Library Granted Patent US 10,242,270
Granted Patent B2
US 10,242,270 · App. 15/449,070 · Granted Mar 26, 2019

Methods and apparatus to count people in images

Inventors: Venugopal Srinivasan (Tarpon Springs, FL); Padmanabhan Soundararajan (Tampa, FL)
Assignee: The Nielsen Company (US), LLC
G06K9/00771G06K9/00221G06K9/00228G06T7/20G06T7/70G06T2207/10016G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,242,270
App. No.
15/449,070
Granted
Mar 26, 2019
Kind
B2
Abstract

Methods and apparatus to determine a blob people count in images are disclosed. An example method includes forming blobs based on whitened pixels of detected face rectangles in pairs of frames of a plurality of image frame captured over a period of time. The method further calculates locations of the centers of the blobs in each frame of the first one of the pairs of frames. The method further eliminates blobs in a first frame of the first one of the pairs of frames having locations within one or more blobs in the second frame of the first one of the pairs of frames. The method further adds the number of remaining blobs from the first frame to the number of blobs in the second frame of the first one of the pairs of frames to form a blob people count.

Claims (38)

1. An apparatus, comprising:

a blob creator to form blobs based on whitened pixels of detected face rectangles in pairs of frames of a plurality of image frames captured over a period of time, respective ones of the pairs of frames including one frame obtained with a first image sensor and another frame obtained with a second image sensor;

a center of gravity calculator to calculate locations of centers of the blobs in each frame of the pairs of frames;

a location analyzer to eliminate respective ones of the blobs from a first set of blobs in a first frame of a first one of the pairs of frames, the respective ones of the blobs to be eliminated from the first set having respective locations within an overlap region corresponding to an intersection of a first field of view of the first image sensor and a second field of view of the second image sensor; and

an adder to add a number of blobs remaining in the first set of blobs of the first frame of the first one of the pairs of frames to a number of blobs in a second set of blobs of a second frame of the first one of the pairs of frames to detect a blob people count represented in the first one of the pairs of frames.

2. The apparatus as defined in claim 1 , further including a pixel whitener to whiten pixels in the detected face rectangles in the pairs of frames.

3. The apparatus as defined in claim 2 , wherein the pixel whitener blackens pixels not in the detected face rectangles in the pairs of frames.

4. The apparatus as defined in claim 1 , wherein the plurality of image frames are captured in a media exposure environment, and further including associating the blob people count with media identifying information detected in the media exposure environment during the period of time.

5. The apparatus as defined in claim 1 , wherein the blob creator forms the blobs by:

overlaying the whitened pixels from the pairs of frames of the plurality of image frames captured over the period of time; and

combining the overlaid whitened pixels to form the blobs.

6. The apparatus as defined in claim 1 , further including a discrepancy resolver to resolve a discrepancy between the blob people count and a group people count for the period of time.

7. The apparatus as defined in claim 6 , wherein the discrepancy resolver uses data associated with preceding periods of time to choose one of the group people count or the blob people count as a people tally for the period of time.

8. A tangible computer readable storage medium comprising instructions that, when executed, cause a processor to at least:

form blobs based on whitened pixels of detected face rectangles in pairs of frames of a plurality of image frames captured over a period of time, respective ones of the pair of frames including one frame obtained with a first image sensor and another frame obtained with a second image sensor;

calculate locations of centers of the blobs in each frame of the pairs of frames;

eliminate respective ones of the blobs from a first set of blobs in a first frame of a first one of the pairs of frames, the respective ones of the blobs to be eliminated from the first set having locations within an overlap region corresponding to an intersection of a first field of view of the first image sensor and a second field of view of the second image sensor; and

add a number of blobs remaining in the first set of blobs of the first frame of the first one of the pairs of frames to a number of blobs in a second set of blobs of a second frame of the first one of the pairs of frames to detect a blob people count represented in the first one of the pairs of frames.

9. The tangible computer readable storage medium as defined in claim 8 , wherein the instructions, when executed, further cause the processor to whiten pixels in the detected face rectangles in the pairs of frames.

10. The tangible computer readable storage medium as defined in claim 9 , wherein the instructions, when executed, further cause the processor to blacken pixels not in the detected face rectangles in the pairs of frames.

11. The tangible computer readable storage medium as defined in claim 8 , wherein the plurality of image frames are captured in a media exposure environment, and the instructions further cause the processor to associate the blob people count with media identifying information detected in the media exposure environment during the period of time.

12. The tangible computer readable storage medium as defined in claim 8 , wherein the instructions, when executed, further cause the processor to create the blobs by:

overlaying the whitened pixels from the pairs of frames of the plurality of image frames captured over the period of time; and

combining the overlaid whitened pixels to form the blobs.

13. The tangible computer readable storage medium as defined in claim 8 , wherein the instructions, when executed, further cause the processor to resolve a discrepancy between the blob people count and a group people count for the period of time.

14. The tangible computer readable storage medium as defined in claim 13 , wherein the instructions, when executed, further cause the processor to use data associated with preceding periods of time to resolve the discrepancy by choosing one of the group people count or the blob people count as a people tally for the period of time.

15. A method, comprising:

forming, by executing an instruction with a processor, blobs based on whitened pixels of detected face rectangles in pairs of frames of a plurality of image frame captured over a period of time, respective ones of the pair of frames including one frame obtained with a first image sensor and another frame obtained with a second image sensor;

calculating, by executing an instruction with the processor, locations of centers of the blobs in each frame of the pairs of frames;

eliminating, by executing an instruction with the processor, blobs from a first set of blobs in a first frame of a first one of the pairs of frames, the respective ones of the blobs to be eliminated from the first set having respective locations within an overlap region corresponding to an intersection of a first field of view of the first image sensor and a second field of view of the second image sensor; and

adding, by executing an instruction with the processor, a number of blobs remaining in the first set of blobs of the first frame of the first one of the pairs of frames to a number of blobs in a second set of blobs of a second frame of the first one of the pairs of frames to form a blob people count represented in the first one of the pairs of frames.

16. The method as defined in claim 15 , further including whitening pixels in the detected face rectangles in the pairs of frames and blackening pixels not in the detected face rectangles in the pairs of frames.

17. The method as defined in claim 15 , wherein the plurality of image frames are captured in a media exposure environment, and further including associating the blob people count with media identifying information detected in the media exposure environment during the period of time.

18. The method as defined in claim 15 , wherein the blobs are formed by:

overlaying the whitened pixels from the pairs of frames of the plurality of image frames captured over the period of time; and

combining the overlaid whitened pixels to form the blobs.

19. The method as defined in claim 15 , further including resolving discrepancies between the blob people count and a group people count for the period of time.

20. The method as defined in claim 19 , wherein the discrepancies are resolved using data associated with preceding periods of time to select one of the group people count or the blob people count as a people tally for the period of time.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 8, 2017
From: SRINIVASAN, VENUGOPAL; SOUNDARARAJAN, PADMANABHAN
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 041508/0030 →
Continuity (3)
Continuation 14732107 · Jun 5, 2015
Continuation 13434337 · Mar 29, 2012
Related Publication 20170177948A1 · Jun 22, 2017
Cited By (2)
US 12,425,688 US 12,432,415