DETECTED OBJECT TRACKER FOR A VIDEO ANALYTICS SYSTEM
Techniques are disclosed which provide a detected object tracker for a video analytics system. As disclosed, the detected object tracker provides a robust foreground object tracking component for a video analytics system which allow other components of the video analytics system to more accurately evaluate the behavior of a given object (as well as to learn to identify different instances or occurrences of the same object) over time. More generally, techniques are disclosed for identifying what pixels of successive video frames depict the same foreground object. Logic implementing certain functions of the detected object tracker can be executed on either a conventional processor (e.g., a CPU) or a hardware acceleration processing device (e.g., a GPU), allowing multiple camera feeds to be evaluated in parallel.
1 . A computer-implemented method for tracking foreground objects depicted in a scene, the method comprising:
launching a tracker component of a video analytics system executed on one or more processing units of the video analytics system;
performing, by the tracker component, a geometric matching between a first set of bounded regions associated with a current video frame of the scene and a second set of bounded regions associated with a previous video frame of the scene, wherein the geometric matching assigns one or more of the first set of bounded regions to a known set of foreground objects and one or more of the first set of bounded regions to a discovered set of foreground objects;
retrieving, by the tracker component, a third set of bounded regions, each corresponding to a missing foreground objects tracked in one or more previous video frames of the scene; and
processing, on one or more processing pipelines of a hardware acceleration device, one or more of the bounded regions in the known set of foreground objects, in the discovered set of foreground objects, and in the missing set of foreground objects to determine a set of tracked foreground objects observed by the video analytics system in the current frame.
2 . The method of claim 1 , further comprising:
extending a trajectory of each tracked foreground object observed by the video analytics system in the current frame based at least on at least a corresponding set of appearance values of pixels depicting the tracked foreground object in the current frame.
3 . The method of claim 1 , wherein the hardware acceleration device comprises one or more graphics processing units (CPUs).
4 . The method of claim 1 , wherein each bounded region in the first set of bounded regions comprises an ellipse bounding a region of pixels classified as depicting scene foreground in the current frame and wherein each bounded region in the second set of bounded regions comprises an ellipse bounding a region of pixels classified as depicting a tracked foreground object in the previous frame.
5 . The method of claim 4 , wherein the geometric matching comprises comparing a size and orientation of a first ellipse in the first set of bounded regions with one or more second ellipses in the second set of bounded regions.
6 . The method of claim 4 , wherein processing one or more of the bounded regions in the known set of foreground objects comprises:
upon determining a geometric match between the first ellipse and a given one of the second ellipses, performing, on the hardware acceleration device, an appearance matching between pixels of the first ellipse and pixels of the given second ellipse.
7 . The method of claim 4 , wherein processing one or more of the bounded regions in the missing set of foreground objects, comprises:
performing, on the hardware acceleration device, an appearance matching between pixels of a first one of the missing foreground objects and appearance values at a plurality of locations in the current frame determined using a location particle filter (LOPART), wherein the LOPART generates each of the plurality of locations based on at least motion data associated with the first missing foreground object and a measure of randomness; and
upon determining an appearance match between the first missing foreground object and a given one of the locations determined using a location particle filter, generating, by a size and orientation particle filter, an ellipse to bound the missing foreground object in the current frame.
8 . A computer-readable storage medium containing a program, which when executed on a processor, performs an operation for tracking foreground objects depicted in a scene, the method comprising:
launching a tracker component of a video analytics system executed on one or more processing units of the video analytics system;
performing, by the tracker component, a geometric matching between a first set of bounded regions associated with a current video frame of the scene and a second set of bounded regions associated with a previous video frame of the scene, wherein the geometric matching assigns one or more of the first set of bounded regions to a known set of foreground objects and one or more of the first set of bounded regions to a discovered set of foreground objects;
retrieving, by the tracker component, a third set of bounded regions, each corresponding to a missing foreground objects tracked in one or more previous video frames of the scene; and
processing, on one or more processing pipelines of a hardware acceleration device, one or more of the bounded regions in the known set of foreground objects, in the discovered set of foreground objects, and in the missing set of foreground objects to determine a set of tracked foreground objects observed by the video analytics system in the current frame.
9 . The computer-readable storage medium of claim 8 , wherein the operation further comprises:
extending a trajectory of each tracked foreground object observed by the video analytics system in the current frame based on a corresponding set of appearance values of the tracked foreground object.
10 . The computer-readable storage medium of claim 8 , wherein the hardware acceleration device comprises one or more graphics processing units (GPU).
11 . The computer-readable storage medium of claim 8 , wherein each bounded region in the first set of bounded regions comprises an ellipse bounding a region of pixels classified as depicting scene foreground in the current frame and wherein each bounded region in the second set of bounded regions comprises an ellipse bounding a region of pixels classified as depicting scene foreground in the previous frame.
12 . The computer-readable storage medium of claim 11 , wherein the geometric matching comprises comparing a size and orientation of a first ellipse in the first set of bounded regions with one or more second ellipses in the second set of bounded regions.
13 . The computer-readable storage medium of claim 11 , wherein processing one or more of the bounded regions in the known set of foreground objects comprises:
upon determining a geometric match between the first ellipse and a given one of the second ellipses, performing, on the hardware acceleration device, an appearance matching between pixels of the first ellipse and pixels of the given second ellipse.
14 . The computer-readable storage medium of claim 11 , wherein processing one or more of the bounded regions in the missing set of foreground objects, comprises:
performing, on the hardware acceleration device, an appearance matching between pixels of a first one of the missing foreground objects and appearance values at a plurality of locations in the current frame determined using a location particle filter (LOPART), wherein the LOPART generates each of the plurality of locations based on at least motion data associated with the first missing foreground object and a measure of randomness; and
upon determining an appearance match between the first missing foreground object and a given one of the locations determined using a location particle filter, generating, by a size and orientation particle filter, an ellipse to bound the missing foreground object in the current frame.
15 . A video analytics system, comprising:
a video input source configured to provide a sequence of video frames, each depicting a scene;
at least one graphics processing unit; and
at least one central processing unit (CPU);
a memory containing a tracker component of the video analytics system, which, when executed on the CPU performs an operation for tracking foreground objects depicted in the scene, the operation comprising:
performing, by the tracker component, a geometric matching between a first set of bounded regions associated with a current video frame of the scene and a second set of bounded regions associated with a previous video frame of the scene, wherein the geometric matching assigns one or more of the first set of bounded regions to a known set of foreground objects and one or more of the first set of bounded regions to a discovered set of foreground objects,
retrieving, by the tracker component, a third set of bounded regions, each corresponding to a missing foreground objects tracked in one or more previous video frames of the scene, and
processing, on one or more processing pipelines of a hardware acceleration device, one or more of the bounded regions in the known set of foreground objects, in the discovered set of foreground objects, and in the missing set of foreground objects to determine a set of tracked foreground objects observed by the video analytics system in the current frame.
16 . The system of claim 15 , wherein the operation further comprises, extending a trajectory of each tracked foreground object observed by the video analytics system in the current frame based on a corresponding set of appearance values of the tracked foreground object.
17 . The system of claim 15 , wherein each bounded region in the first set of bounded regions comprises an ellipse bounding a region of pixels classified as depicting scene foreground in the current frame and wherein each bounded region in the second set of bounded regions comprises an ellipse bounding a region of pixels classified as depicting scene foreground in the previous frame.
18 . The system of claim 17 , wherein the geometric matching comprises comparing a size and orientation of a first ellipse in the first set of bounded regions with one or more second ellipses in the second set of bounded regions.
19 . The system of claim 17 , wherein processing one or more of the bounded regions in the known set of foreground objects comprises:
upon determining a geometric match between the first ellipse and a given one of the second ellipses, performing, on the hardware acceleration device, an appearance matching between pixels of the first ellipse and pixels of the given second ellipse.
20 . The system of claim 17 , wherein processing one or more of the bounded regions in the missing set of foreground objects, comprises:
performing, on the hardware acceleration device, an appearance matching between pixels of a first one of the missing foreground objects and appearance values at a plurality of locations in the current frame determined using a location particle filter (LOPART), wherein the LOPART generates each of the plurality of locations based on at least motion data associated with the first missing foreground object and a measure of randomness; and
upon determining an appearance match between the first missing foreground object and a given one of the locations determined using a location particle filter, generating, by a size and orientation particle filter, an ellipse to bound the missing foreground object in the current frame.