IP Library Granted Patent US 7,203,338
Granted Patent B2
US 7,203,338 · App. 10/538,483 · Granted Apr 10, 2007

Methods and apparatus to count people appearing in an image

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,203,338
App. No.
10/538,483
Granted
Apr 10, 2007
Kind
B2
Abstract

Methods and apparatus to count people appearing in an image are disclosed. One disclosed method reduces objects appearing in a series of images to one or more blobs; for each individual image in a set of the images of the series of images, represents the one or more blobs in the individual image by one or more symbols in a histogram; and analyzes the symbols appearing in the histogram to count the people in the image.

Claims (67)

1. An apparatus to count people in an image comprising:

a motion detector to compare at least two images to detect motion occurring between the at least two images to develop a difference image;

a shape outliner to draw at least one shape based on the difference image; and

a blob discriminator to determine if the at least one shape represents a person;

wherein the blob discriminator comprises:

a center locator to identify a center of the at least one shape;

a center comparator to add a symbol representative of the center of the at least one shape to a histogram; and

a threshold counter to count symbols in the histogram exceeding a predetermined threshold as persons.

2. An apparatus as defined in claim 1 wherein the motion detector compares at least three images to develop at least two difference images.

3. An apparatus as defined in claim 2 further comprising an image amalgamator to develop an amalgamated image from the at least two difference images.

4. An apparatus as defined in claim 3 wherein the shape outliner draws the at least one shape by joining all points within the amalgamated image that satisfy a predetermined constraint into the at least one shape.

5. A method as defined in claim 4 wherein the predetermined constraint requires a distance between all points in the at least one shape to be less than a predetermined distance.

6. An apparatus as defined in claim 1 further comprising a non-human filter to eliminate a non-human shape from the at least one shape.

7. An apparatus as defined in claim 6 wherein the non-human filter eliminates the non-human shape based on at least one of a location of the non-human shape and a size of the non-human shape.

8. An apparatus as defined in claim 1 wherein if the center of the at least one shape substantially corresponds to an existing center in the histogram, the center comparator adds the symbol representative of the center of the at least one shape to a symbol representing the existing center in the histogram.

9. An apparatus as defined in claim 1 further comprising an energy detector to compare a value indicative of the motion occurring between the two images to an energy threshold, and to cause the threshold counter to count the symbols in the histogram exceeding the predetermined threshold if the value exceeds the energy threshold.

10. An apparatus as defined in claim 8 further comprising a false motion detector to eliminate a non-growing symbol from the histogram.

11. A method of determining a number of people within at least one image comprising:

(a) determining at least one difference image between at least two images;

(b) developing at least one shape from the at least one difference image;

(c) identifying a geometric parameter of the at least one shape;

(d) adding a symbol having a predetermined size and indicative of the geometric parameter of the at least one shape to a histogram;

(e) repeating (a)–(d); and

(f) if any symbol in the histogram grows beyond a predetermined threshold, counting the symbol as a person.

12. A method as defined in claim 11 wherein determining at least one difference image between at least two of the images comprises:

(a) determining a first difference image between a first image and a second image; and

(b) determining a second difference image between the second image and a third image.

13. A method as defined in claim 12 further comprising developing an amalgamated image from the first and second difference images.

14. A method as defined in claim 13 wherein developing at least one shape from the at least one difference image comprises developing at least one shape from the amalgamated image.

15. A method as defined in claim 11 wherein identifying at least one geometric parameter of the at least one shape comprises identifying a center of a first shape and a center of a second shape.

16. A method as defined in claim 15 wherein adding a symbol having a predetermined size and indicative of the geometric parameter of the at least one shape to the histogram comprises adding a first symbol indicative of the center of the first shape to the histogram and adding a second symbol indicative of the center of the second shape to the histogram.

17. A method as defined in claim 16 wherein repeating (a)–(d) comprises stacking a third symbol on the first symbol if a second difference image contains a shape having a center that substantially corresponds to the center of the first shape.

18. A method as defined in claim 11 further comprising excluding a shape from a group of possible human shapes based on a test.

19. A method as defined in claim 18 wherein the test comprises at least one of a location test and a size test.

20. A method as defined in claim 11 further comprising identifying an energy value associated with the at least one difference image, and, performing (f) if the energy value exceeds a predetermined threshold.

21. A method as defined in claim 11 wherein developing at least one shape from the at least one difference image comprises executing a convex hull process.

22. A method as defined in claim 11 wherein, if any symbol does not grow within a predetermined length of time, it is eliminated from the histogram.

23. A method of counting people appearing in a digital image comprising:

reducing objects appearing in a series of images to one or more blobs;

for each individual image in a set of the images of the series of images, representing the one or more blobs in the individual image by one or more symbols in a histogram; and

analyzing the symbols appearing in the histogram to count the people in the image

wherein representing one or more blobs in the individual image by a symbol in the histogram further comprises:

identifying one or more centers of the one or more blobs; and

placing the one or more symbols in the histogram at one or more locations indicative of the one or more centers of the one or more blobs; and

wherein each of the one or more symbols has a predetermined size and, further corn comprising, if the center of a first blob in the one or more blobs substantially corresponds to a center of a second blob in the one or more blobs, adding a symbol corresponding to the first blob to a symbol corresponding to the second blob in the histogram.

24. A method as defined in claim 23 wherein reducing the objects appearing in the series of images to one or more blobs comprises creating the one or more blobs using a convex hull program.

25. A method as defined in claim 23 further comprising, if no symbols are added to a third symbol appearing in the histogram during a period in which a number of images are analyzed, removing the third symbol from the histogram.

26. A method as defined in claim 25 wherein analyzing the symbols appearing in the histogram to count the people in the digital image comprises, counting the symbols appearing in the histogram having a size greater than a threshold.

27. A method as defined in claim 23 wherein the series of images comprises a series of amalgamated images.

28. A method as defined in claim 27 wherein the series of amalgamated images are derived from a series of difference images, the series of difference images being derived from a sequence of original images including the digital image.

29. A machine readable medium storing machine readable instructions which, when executed, cause a machine to:

(a) determine at least one difference image between at least two images;

(b) develop at least one shape from the at least one difference image;

(c) identify a geometric parameter of the at least one shape;

(d) add a symbol having a predetermined size and indicative of the geometric parameter of the at least one shape to a histogram;

(e) repeat (a)–(d); and

(f) if any symbol in the histogram grows beyond a predetermined threshold, count the symbol as a person.

30. An apparatus to count people appearing in a digital image comprising:

a processor;

a memory storing computer readable instructions which, when executed, cause the processor to:

reduce objects appearing in a series of images to one or more blobs;

for each individual image in a set of the images of the series of images, represent the one or more blobs in the individual image by one or more symbols in a histogram; and

analyze the symbols appearing in the histogram to count the people in the image;

wherein the machine readable instructions cause the processor to represent the one or more blobs in the individual image by a symbol in the histogram by;

identifying one or more centers of the one or more blobs; and

placing the one or more symbols in the histogram at one or more locations indicative of the one or more centers of the one or more blobs; and

wherein each of the one or more symbols has a predetermined size and, wherein, if the center of a first blob in the one or more blobs substantially corresponds to a center of a second blob In the one or more blobs, the machine readable instructions cause the processor to add a symbol corresponding to the first blob to a symbol corresponding to the second blob in the histogram.

Assignments (10)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
RELEASE (REEL 018207 / FRAME 0607) Recorded Oct 13, 2022
From: CITIBANK, N.A.
To: VNU MARKETING INFORMATION, INC.; THE NIELSEN COMPANY (US), LLC
Reel/Frame 061749/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
MERGER Recorded Jun 29, 2009
From: NIELSEN MEDIA RESEARCH, LLC (FORMERLY KNOWN AS NIELSEN MEDIA RESEARCH, INC.) A DELAWARE CORPORATION
To: NIELSEN COMPANY (US), LLC, THE
Reel/Frame 022892/0544 →
SECURITY AGREEMENT Recorded Sep 7, 2006
From: NIELSEN MEDIA RESEARCH, INC.; AC NIELSEN (US), INC.; BROADCAST DATA SYSTEMS, LLC; VNU MARKETING INFORMATION, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 018207/0607 →