Method and apparatus for improved video surveillance through classification of detected objects

1. A computer-implemented method for classifying a moving object in a field of view, comprising:

using a processor to perform the steps of:

receiving a sequence of scene imagery representing said field of view from at least one image capturing device;

identifying said moving object in said sequence of scene imagery;

classifying said moving object in accordance with one or more extracted spatio-temporal features of said moving object, said classifying including comparing the moving object's spatio-temporal features to the spatio-temporal features of one or more learned objects; and

generating an alert in response to a newly detected moving object by comparing the extracted spatio-temporal features to one or more learned events, the learned events being stored as histograms of motion-based spatio-temporal features for every pixel in an image of the learned event, the histograms representing the probability distribution of spatio-temporal features due to normal activity in the field of view.

2. The method of claim 1 , wherein said receiving further comprises receiving one or more motion detection masks associated with said moving object.

3. The method of claim 1 , wherein said identifying comprises:

separating said moving object from one or more other moving objects in said sequence of scene imagery; and

performing short-term object tracking for said moving object.

4. The method of claim 3 , wherein said separating comprises: processing one or more motion detection masks associated with said moving object in accordance with a connected component analysis.

5. The method of claim 3 , wherein said performing comprises: tracking said moving object over two or more frames of said sequence of scene imagery in accordance with optical flow analysis.

6. The method of claim 1 , wherein said classifying comprises:

extracting a spatial distribution of said moving object's residual motion over a bounding box of said moving object;

extracting one or more dominant temporal frequencies of said moving object over said bounding box; and

producing, in accordance with said spatial distribution and said one or more dominant temporal frequencies, a feature/activity image for said moving object.

7. The method of claim 6 , wherein said feature/activity image is substantially equal in size to said bounding box.

8. The method of claim 6 , wherein said feature/activity image comprises a plurality of pixels, each of said plurality of pixels providing a respective temporal motion amplitude and a respective frequency at a plurality of spatial scales.

9. The method of claim 6 , further comprising: performing a first classification stage in which said moving object is classified, based on said feature/activity image, as a either a rigid object or a dynamic object.

10. The method of claim 9 , further comprising: performing a second classification stage in which said moving object is classified, based on said feature/activity image, as either a human or an animal, if said moving object is determined to be a dynamic object in said first classification stage.

11. The method of claim 10 , wherein said first classification stage and said second classification stage are performed in accordance with a cascade of two two-class support vector machine classifiers.

12. The method of claim 1 , further comprising: analyzing said classified moving object to determine whether behavior of said moving object should trigger an alarm.

13. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of a method for classifying a moving object in a field of view, comprising:

receiving a sequence of scene imagery representing said field of view;

identifying said moving object in said sequence of scene imagery;

classifying said moving object in accordance with one or more extracted spatio-temporal features of said moving object, said classifying including comparing the moving object's spatio-temporal features to the spatio-temporal features of one or more learned objects; and

generating an alert in response to a newly detected moving object by comparing the extracted spatio-temporal features to one or more learned events, the learned events being stored as histograms of motion-based spatio-temporal features for every pixel in an image of the learned event, the histograms representing the probability distribution of spatio-temporal features due to normal activity in the field of view.

14. The computer-readable medium of claim 13 , wherein said identifying comprises:

separating said moving object from one or more other moving objects in said sequence of scene imagery; and

performing short-term object tracking for said moving object.

15. The computer-readable medium of claim 14 , further comprising:

tracking said moving object over two or more frames of said sequence of scene imagery in accordance with optical flow analysis.

16. The computer-readable medium of claim 13 , wherein said classifying comprises:

extracting a spatial distribution of said moving object's residual motion over a bounding box of said moving object;

extracting one or more dominant temporal frequencies of said moving object over said bounding box; and

producing, in accordance with said spatial distribution and said one or more dominant temporal frequencies, a feature/activity image for said moving object.

17. The computer-readable medium of claim 16 , further comprising: performing a first classification stage in which said moving object is classified, based on said feature/activity image, as a either a rigid object or a dynamic object.

18. The computer-readable medium of claim 17 , further comprising: performing a second classification stage in which said moving object is classified, based on said feature/activity image, as either a human or an animal, if said moving object is determined to be a dynamic object in said first classification stage.

19. The computer readable medium of claim 13 , further comprising: analyzing said classified moving object to determine whether behavior of said moving object should trigger an alarm.

20. An apparatus for a moving object in a field of view, comprising:

means for receiving a sequence of scene imagery representing said field of view;

means for identifying said moving object in said sequence of scene imagery;

means for classifying said moving object in accordance with one or more extracted spatio-temporal features of said moving object, said means for classifying including means for comparing the moving object's spatio-temporal features to the spatio-temporal features of one or more learned objects; and

means for generating an alert in response to a newly detected moving object by comparing the extracted spatio-temporal features to one or more learned events, the learned events being stored as histograms of motion-based spatio-temporal features for every pixel in an image of the learned event, the histograms representing the probability distribution of spatio-temporal features due to normal activity in the field of view.