DETECTED OBJECT TRACKER FOR A VIDEO ANALYTICS SYSTEM
Techniques are disclosed which provide a detected object tracker for a video analytics system. As disclosed, the detected object tracker provides a robust foreground object tracking component for a video analytics system which allow other components of the video analytics system to more accurately evaluate the behavior of a given object (as well as to learn to identify different instances or occurrences of the same object) over time. More generally, techniques are disclosed for identifying what pixels of successive video frames depict the same foreground object. Logic implementing certain functions of the detected object tracker can be executed on either a conventional processor (e.g., a CPU) or a hardware acceleration processing device (e.g., a GPU), allowing multiple camera feeds to be evaluated in parallel.
1 . A computer-implemented method for tracking foreground objects depicted in a scene, the method comprising:
receiving a background/foreground BG/FG segmentation of a current video frame, wherein the current video frame includes one or more appearance values for each of a plurality of pixels and wherein the BG/FG segmentation classifies one or more of the pixels as depicting either scene foreground or scene background;
for each region of pixels in the current frame classified as depicting scene foreground, determining an ellipse to bound the region;
classifying at least a first one of the ellipses as corresponding to a first known foreground object tracked in at least a prior frame; and
extending a trajectory of the first known foreground object based on the appearance values of the pixels bound by the first ellipse.
2 . The method of claim 1 , wherein classifying the first ellipse as corresponding to the first known foreground object comprises matching one or more of a size, orientation, and location of the first ellipse with an ellipse generated for the first known foreground object tracked in at least the prior frame.
3 . The method of claim 2 , wherein classifying the first ellipse as corresponding to the first known foreground object further comprises, comparing a weighted color distribution (WCD) generated using the appearance values of pixels bound by the first ellipse with a WCD generated using appearance values of pixels from the ellipse generated for the first known foreground object tracked in at least the prior frame.
4 . The method of claim 1 , wherein extending the trajectory of the first known foreground object comprises generating one or more of appearance data, position data, velocity data, and direction data.
5 . The method of claim 1 , further comprising, classifying at least a second ellipse as depicting a new foreground object within the scene.
6 . The method of claim 5 , further comprising, initializing a trajectory for the new foreground object.
7 . The method of claim 5 , further comprising, upon determining a size and orientation of the second ellipse matches an ellipse in a missing set:
reclassifying the second ellipse as corresponding to a second known foreground object; and
extending a trajectory of the second known foreground object based on the appearance values of the pixels bound by the second ellipse.
8 . The method of claim 1 , further comprising:
identifying an ellipse corresponding to a foreground object tracked in a prior frame, wherein the identified ellipse was not matched to any of the ellipses determined for the current frame; and
adding the identified ellipse to a set of missing ellipses.
9 . A computer-readable storage medium containing a program, which when executed on a processor, performs an operation for tracking foreground objects depicted in a scene, the operation comprising:
receiving a background/foreground BG/FG segmentation of a current video frame, wherein the current video frame includes one or more appearance values for each of a plurality of pixels and wherein the BG/FG segmentation classifies one or more of the pixels as depicting either scene foreground or scene background;
for each region of pixels in the current frame classified as depicting scene foreground, determining an ellipse to bound the region;
classifying at least a first one of the ellipses as corresponding to a first known foreground object tracked in at least a prior frame; and
extending a trajectory of the first known foreground object based on the appearance values of the pixels bound by the first ellipse.
10 . The computer-readable storage medium of claim 9 , wherein classifying the first ellipse as corresponding to the first known foreground object comprises matching one or more of a size, orientation, and location of the first ellipse with an ellipse generated for the first known foreground object tracked in at least the prior frame.
11 . The computer-readable storage medium of claim 10 , wherein classifying the first ellipse as corresponding to the first known foreground object further comprises, comparing a weighted color distribution (WCD) generated using the appearance values of pixels bound by the first ellipse with a WCD generated using appearance values of pixels from the ellipse generated for the first known foreground object tracked in at least the prior frame.
12 . The computer-readable storage medium of claim 9 , wherein extending the trajectory of the first known foreground object comprises generating one or more of appearance data, position data, velocity data, and direction data.
13 . The computer-readable storage medium of claim 9 , wherein the operation further comprises, classifying at least a second ellipse as depicting a new foreground object within the scene.
14 . The computer-readable storage medium of claim 13 , wherein the operation further comprises, initializing a trajectory for the new foreground object.
15 . The computer-readable storage medium of claim 13 , wherein the operation further comprises, upon determining a size and orientation of the second ellipse matches an ellipse in a missing set:
reclassifying the second ellipse as corresponding to a second known foreground object; and
extending a trajectory of the second known foreground object based on the appearance values of the pixels bound by the second ellipse.
16 . The computer-readable storage medium of claim 9 , wherein the operation further comprises:
identifying an ellipse corresponding to a foreground object tracked in a prior frame, wherein the identified ellipse was not matched to any of the ellipses determined for the current frame; and
adding the identified ellipse to a set of missing ellipses.
17 . A system, comprising:
a video input source configured to provide a sequence of video frames, each depicting a scene;
a central processing unit (CPU); and
a memory containing a program, which, when executed on the CPU is configured to perform an operation for tracking foreground objects depicted in a scene, the operation comprising:
receiving a background/foreground BG/FG segmentation of a current video frame, wherein the current video frame includes one or more appearance values for each of a plurality of pixels and wherein the BG/FG segmentation classifies one or more of the pixels as depicting either scene foreground or scene background,
for each region of pixels in the current frame classified as depicting scene foreground, determining an ellipse to bound the region,
classifying at least a first one of the ellipses as corresponding to a first known foreground object tracked in at least a prior frame, and
extending a trajectory of the first known foreground object based on the appearance values of the pixels bound by the first ellipse.
18 . The system of claim 17 , wherein classifying the first ellipse as corresponding to the first known foreground object comprises matching one or more of a size, orientation, and location of the first ellipse with an ellipse generated for the first known foreground object tracked in at least the prior frame.
19 . The system of claim 18 , wherein classifying the first ellipse as corresponding to the first known foreground object further comprises, comparing a weighted color distribution (WCD) generated using the appearance values of pixels bound by the first ellipse with a WCD generated using appearance values of pixels from the ellipse generated for the first known foreground object tracked in at least the prior frame.
20 . The system of claim 17 , wherein extending the trajectory of the first known foreground object comprises generating one or more of appearance data, position data, velocity data, and direction data.
21 . The system of claim 17 , wherein the operation further comprises, classifying at least a second ellipse as depicting a new foreground object within the scene.
22 . The system of claim 21 , wherein the operation further comprises, initializing a trajectory for the new foreground object.
23 . The system of claim 21 , wherein the operation further comprises, upon determining a size and orientation of the second ellipse matches an ellipse in a missing set:
reclassifying the second ellipse as corresponding to a second known foreground object; and
extending a trajectory of the second known foreground object based on the appearance values of the pixels bound by the second ellipse.
24 . The system of claim 17 , wherein the operation further comprises:
identifying an ellipse corresponding to a foreground object tracked in a prior frame, wherein the identified ellipse was not matched to any of the ellipses determined for the current frame; and
adding the identified ellipse to a set of missing ellipses.