IP Library › Granted Patent US 11,151,365
Granted Patent B2
US 11,151,365 · App. 16/436,008 · Granted Oct 19, 2021

People detection system with feature space enhancement

Inventors: Sumandeep Banerjee (Karnataka, IN); Subrat Panda (Karnataka, IN); Doney Alex (Karnataka, IN)
Assignee: Capillary Technologies International PTE LTD
G06K9/00362G06K9/00718G06K9/00744G06K9/00778G06K9/346G06K9/6256
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,151,365
App. No.
16/436,008
Granted
Oct 19, 2021
Kind
B2
Abstract

A people detection system with feature space enhancement is provided. The system includes a memory having computer-readable instructions stored therein. The system includes a processor configured to access a plurality of video frames captured using one or more overhead video cameras installed in a space and to extract one or more raw images of the space from the plurality of video frames. The processor is further configured to process the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples. The positive image samples include images having one or more persons present within the space and the negative image samples comprise images without the persons. The processor is configured to apply at least one of a crop factor and a resize factor to the positive and the negative image samples to generate curated positive and negative image samples and to detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples.

Claims (74)

1. A people detection system with feature space enhancement, the system comprising:

a memory having computer-readable instructions stored therein;

a processor configured to:

access a plurality of video frames captured using one or more overhead video cameras installed in a space;

extract one or more raw images of the space from the plurality of video frames;

process the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;

apply at least one of a crop factor or a resize factor to the positive and the negative image samples to generate curated positive and negative image samples;

detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples, wherein the detection model comprises a plurality of classifiers;

iteratively train the detection model using feature vectors corresponding to the curated positive and negative image samples;

perform negative sample mining using a partially trained detection model; and

adjust the crop factor or resize factor.

2. The people detection system of claim 1 , wherein the processor is further configured to execute the computer-readable instructions to:

annotate the raw images to identify one or more persons in the raw images;

extract annotated portions of the raw images and rotate the extracted annotated portions at a plurality of rotation angles to generate rotated image samples; and

combine the rotated image samples with a plurality of backgrounds to generate the plurality of positive image samples.

3. The people detection system of claim 2 , wherein the processor is further configured to execute the computer-readable instructions to utilize pyramid blending for generating the positive image samples, wherein pyramid blending substantially minimizes effects of boundary artifacts.

4. The people detection system of claim 1 , wherein the processor is further configured to execute the computer-readable instructions to:

generate a plurality of sub-images using the negative image samples; and

select sub-images from the plurality of sub-images having a resolution substantially similar to resolution of the positive image samples.

5. The people detection system of claim 1 , wherein the processor is further configured to execute the computer-readable instructions to:

extract a plurality of channel features using the positive and negative image samples; and

determine the feature vectors using the extracted channel features.

6. The people detection system of claim 1 , wherein the plurality of classifiers of the detection model comprise an AdaBoost classifier for classification and detection of one or more persons present in the space.

7. The people detection system of claim 6 , wherein the processor is further configured to execute the computer-readable instructions to utilize a set of weak classifiers to separate the negative image samples based on a pre-determined set of boundaries.

8. The people detection system of claim 1 , wherein the processor is further configured to execute the computer-readable instructions to:

determine the crop factor to prevent detection of two persons present in close proximity as one person; and

crop the image samples in accordance with the crop factor.

9. The people detection system of claim 8 , wherein the processor is further configured to execute the computer-readable instructions to select the crop factor to remove portions corresponding to shoulders of the two persons present in close proximity.

10. The people detection system of claim 1 , wherein the processor is further configured to execute the computer-readable instructions to determine the resize factor to obtain a desired detection accuracy at a pre-determined frame rate.

11. The people detection system of claim 1 , wherein the processor is further configured to execute the computer-readable instructions to detect location and size of the detected persons present in the space.

12. A people detection system with feature space enhancement, the system comprising:

a plurality of overhead video cameras installed within a space, each of the plurality of overhead video cameras configured to capture real-time video of the space;

a detector communicatively coupled to the plurality of overhead video cameras and configured to detect one or more persons present in the space, wherein the detector comprises:

an image processing module configured to process video frames of the captured video to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;

a feature space estimation module configured to determine one or more feature space parameters, wherein the feature space parameters comprise a crop factor, a resize factor, or combinations thereof; and

a detector module configured to:

apply the one or more feature space parameters to the positive and the negative image samples to generate curated positive and negative image samples;

detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples, wherein the detection model comprises a plurality of classifiers;

iteratively train the detection model using feature vectors corresponding to the curated positive and negative image samples;

perform negative sample mining using a partially trained detection model; and

adjust the crop factor or resize factor.

13. The people detection system of claim 12 , wherein the image processing module is further configured to:

extract raw images from the video frames;

annotate the raw images to identify one or more persons in the raw images;

extract annotated portions of the raw images and rotate the extracted annotated portions at a plurality of rotation angles to generate rotated image samples; and

combine the rotated image samples with a plurality of backgrounds to generate the plurality of positive image samples.

14. The people detection system of claim 12 , wherein the plurality of classifiers of the detection model comprise AdaBoost classifier for classification and detection of one or more persons present in the space.

15. The people detection system of claim 14 , wherein the detection model is further configured to estimate a detection accuracy and a classification accuracy of the detection model.

16. A computer-implemented method for detecting persons in a space, the method comprising:

accessing a plurality of video frames captured using one or more overhead video cameras installed in a space;

extracting one or more raw images of the space from the plurality of video frames;

processing the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;

applying at least one of a crop factor or a resize factor to the positive and the negative image samples to generate curated positive and negative image samples;

detecting one or more persons present in the space using the curated positive and negative image samples;

iteratively training a detection model using the curated positive and negative image samples, wherein the detection model comprises a plurality of classifiers;

performing negative sample mining using a partially trained detection model; and

adjusting the crop factor or resize factor.

17. The computer implemented method of claim 16 , further comprising tracking number of persons entering or exiting the space using Kalman correction and Hungarian Assignment solver.

18. A people detection system with feature space enhancement, the system comprising:

a memory having computer-readable instructions stored therein;

a processor configured to:

access a plurality of video frames captured using one or more overhead video cameras installed in a space;

extract one or more raw images of the space from the plurality of video frames;

process the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;

determine a crop factor to prevent detection of two persons present in close proximity as one person;

apply the crop factor to the positive and the negative image samples to generate curated positive and negative image samples; and

detect one or more persons present in the space using a detection model trained by the curated positive and negative image samples.

19. A computer-implemented method for detecting persons in a space, the method comprising:

accessing a plurality of video frames captured using one or more overhead video cameras installed in a space;

extracting one or more raw images of the space from the plurality of video frames;

processing the one or more raw images to generate a plurality of positive image samples and a plurality of negative image samples, wherein the positive image samples comprise images having one or more persons present within the space and the negative image samples comprise images without the persons;

applying at least one of a crop factor or a resize factor to the positive and the negative image samples to generate curated positive and negative image samples;

detecting one or more persons present in the space using the curated positive and negative image samples; and

tracking number of persons entering or exiting the space using Kalman correction and Hungarian Assignment solver.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 8, 2021
From: BANERJEE, SUMANDEEP; PANDA, SUBRAT; ALEX, DONEY
To: CAPILLARY TECHNOLOGIES INTERNATIONAL PTE LTD
Reel/Frame 054853/0796 →
Priority Claims (1)
IN 201841021909 · Jun 12, 2018 · national
Continuity (1)
Related Publication 20190377940A1 · Dec 12, 2019