Granted Patent B1

US 7,391,907 · App. 10/954,309 · Granted Jun 24, 2008

Abstract

Claims (146)

1. A computer-readable medium comprising software to detect spurious objects, which when executed by a computer system, cause the computer system to perform operations comprising a method of:

detecting an object in frames of a video sequence to obtain a detected object;

tracking said detected object in said frames of said video sequence to obtain a tracked object;

classifying said tracked object as a real object or a spurious object based on at least one of:

a shape consistency metric, including:

extracting an enlarged view (blob) of said object for each of two frames;

creating a mask for each blob;

computing a centroid for each blob;

aligning said masks by said centroids, whereby an aligned mask is created; and

computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including:

measuring a number of pixels in said detected object in each of two frames; and

computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including: computing a color histogram of said object in a plurality of frames, wherein said color histogram for n previous frames is a vector with m elements, where each element of said vector represents a value of said color histogram;

computing a color consistency vector using color histogram vectors for two frames; and

computing a color consistency metric based on said color consistency vector and m, an intensity consistency metric,

a speed consistency metric, including: computing an instantaneous speed of said object in two frames; and computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

a direction of motion consistency metric, including: computing an instantaneous direction of motion of said object in two frames; and computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

2. A computer-readable medium as in claim 1 , further comprising classifying said object based on a size metric including:

determining a current size of said object at a position in a frame;

determining a mean size at said position;

computing a standard deviation of a size at said position; and

computing a size metric from said current size, said mean size and said standard deviation.

3. A computer-readable medium as in claim 1 , wherein classifying said object based on a texture consistency metric comprises:

computing an object texture for said object in two frames; and

computing a texture consistency metric based on a change in said computed object textures in said two frames.

4. A computer-readable medium as in claim 1 , wherein classifying said object based on a intensity consistency metric comprises:

computing an average intensity metric of said object in two frames; and

computing an intensity consistency metric based on a change in said average intensity metric in said two frames.

5. A computer-readable medium as in claim 1 , wherein classifying said object based on a salient motion metric comprises:

determining a position of said object over a window of interest of n frames;

computing a salient motion metric based on a distance between said object position at the start and end times of said window of interest, and a total distance traveled by the object.

6. A computer-readable medium as in claim 1 , wherein classifying said object based on an absolute motion metric comprises:

computing a distance between a position of said object at a start and an end of a time window of n frames.

7. A computer-readable medium as in claim 1 , wherein classifying said object based on a persistent motion metric comprises:

computing a first number of frames in which said object appears over a window of interest having a second number of frames; and

computing a persistent motion metric based on a ratio of said first number of frames to said second number of frames.

8. A computer-readable medium as in claim 1 , wherein said object is classified based on at least two of said classification metrics.

9. A computer-readable medium as in claim 8 , wherein said object is classified based on at least three of said classification metrics.

10. A computer-readable medium as in claim 9 , wherein said object is classified based on at least four of said classification metrics.

11. A computer-readable medium as in claim 1 , wherein said object is classified based on at least one time-averaged classification metric.

12. A computer-readable medium as in claim 1 , wherein said object is classified based on a combination of at least two classification metrics, each said classification metric quantifying a spatial property and/or a temporal property of said tracked object.

13. A computer-readable medium as in claim 12 , wherein said classification metrics are combined using neural networks.

14. A computer-readable medium as in claim 12 , wherein said classification metrics are combined using linear discriminant analysis.

15. A computer-readable medium as in claim 12 , wherein said classification metrics are combined using non-linear weighting.

16. A computer-readable medium as in claim 12 , wherein said classification metrics are time-average weighted over at least one time window.

17. A computer-readable medium as in claim 1 , wherein said method further comprising, if said object is classified as said real object, further classifying said real object as to a type of real object.

18. A computer-readable medium as in claim 1 , wherein said method further comprising if said object is classified as said real object, initiating a triggered response.

19. A computer-readable medium as in claim 1 , wherein said frames are consecutive.

20. A computer-readable medium as in claim 1 , wherein said frames are non-consecutive.

21. An apparatus to perform a method to detect spurious objects, said method comprising:

detecting an object in frames of a video sequence to obtain a detected object;

tracking said detected object in said frames of said video sequence to obtain a tracked object;

classifying said tracked object as a real object or a spurious object based on at least one of:

a shape consistency metric, including:

extracting an enlarged view (blob) of said object for each of two frames;

creating a mask for each blob;

computing a centroid for each blob;

aligning said masks by said centroids, whereby an aligned mask is created; and

computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including:

measuring a number of pixels in said detected object in each of two frames: and

computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including:

computing a color histogram of said object in a plurality of frames, wherein said color histogram for n previous frames is a vector with m elements, where each element of said vector represents a value of said color histogram;

computing a color consistency vector using color histogram vectors for two frames; and

computing a color consistency metric based on said color consistency vector and m,

an intensity consistency metric,

a speed consistency metric, including:

computing an instantaneous speed of said object in two frames; and

computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

a direction of motion consistency metric, including:

computing an instantaneous direction of motion of said object in two frames; and

computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

22. An apparatus as in claim 21 , wherein said object is classified based on at least two of said classification metrics.

23. An apparatus as in claim 22 , wherein said object is classified based on at least three of said classification metrics.

24. An apparatus as in claim 23 , wherein said object is classified based on at least four of said classification metrics.

25. A method to detect spurious objects, comprising:

detecting an object in frames of a video sequence to obtain a detected object;

tracking said detected object in said frames of said video sequence to obtain a tracked object;

classifying said tracked object as a real object or a spurious object based on at least one of the following classification metrics:

a shape consistency metric, including:

extracting an enlarged view (blob) of said object for each of two frames;

creating a mask for each blob;

computing a centroid for each blob;

aligning said masks by said centroids, whereby an aligned mask is created; and

computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including:

measuring a number of pixels in said detected object in each of two frames; and

computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including:

computing a color histogram of said object in a plurality of frames, wherein said color histogram for n previous frames is a vector with m elements, where each element of said vector represents a value of said color histogram;

computing a color consistency vector using color histogram vectors for two frames; and

computing a color consistency metric based on said color consistency vector and m,

an intensity consistency metric,

a speed consistency metric, including:

computing an instantaneous speed of said object in two frames; and

computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

a direction of motion consistency metric, including:

computing an instantaneous direction of motion of said object in two frames, and

computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

26. A method as in claim 25 , wherein said object is classified based on at least two of said classification metrics.

27. A method as in claim 26 , wherein said object is classified based on at least three of said classification metrics.

28. A method as in claim 27 , wherein said object is classified based on at least four of said classification metrics.

29. A system to detect spurious objects, comprising:

means for detecting an object in frames of a video sequence to obtain a detected object;

means for tracking said detected object in said frames of said video sequence to obtain a tracked object;

means for classifying said tracked object as a real object or a spurious object based on at least one of the following classification metrics:

a shape consistency metric, including:

means for extracting an enlarged view (blob) of said object for each of two frames,

means for creating a mask for each blob,

means for computing a centroid for each blob,

means for aligning said masks by said centroids, whereby an aligned mask is created; and

means for computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including:

means for measuring a number of pixels in said detected object in each of two frames; and

means for computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including: means for computing an instantaneous speed of said object in two frames; and computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

an intensity consistency metric,

a speed consistency metric,

a direction of motion consistency metric, including:

means for computing an instantaneous direction of motion of said object in two frames and

means for computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

30. A system as in claim 29 , wherein said object is classified based on at least two of said classification metrics.

31. A system as in claim 30 , wherein said object is classified based on at least three of said classification metrics.

32. A system as in claim 31 , wherein said object is classified based on at least four of said classification metrics.

33. A computer-readable medium as in claim 1 , wherein classifying said object based on an intensity consistency metric comprises:

computing an intensity histogram of said object in a plurality of frames, wherein said intensity histogram for n previous frames is a vector with m elements, where each element represents a value of said intensity histogram;

computing an intensity consistency vector using intensity histogram vectors for two frames; and

computing an intensity consistency metric based on said intensity consistency vector and m.

Assignments (8)