Multi-object tracking using correlation filters in video analytics applications
In various examples, correlation filters may be learned for object tracking based at least on occlusion maps. When learning a correlation filter from an image area, an occlusion map may be applied to the image area that masks, excludes, and/or blurs occlusions of the target object. The correlation filter may be learned from a modified image, thereby reducing or eliminating learning from occlusions while still allowing for learning the target object from exposed portions. The occlusion maps may be generated using a machine learning model, such as a Gaussian Mixture Model (GMM) that is trained (e.g., using the image areas used to learn the correlation filter) using the target object as a background so that occlusions are detected as foreground.
1 . A computer-implemented method comprising:
identifying one or more image areas that depict one or more occluding regions that correspond to one or more occluded portions of one or more objects and depict one or more unoccluded regions of the one or more objects;
generating one or more occlusion maps that identify the one or more occluding regions in the one or more image areas;
learning one or more values of one or more correlation filters from the one or more unoccluded regions in the one or more image areas based at least on reducing or eliminating, from the one or more image areas and using the one or more occlusion maps, the one or more occluding regions; and
detecting the one or more objects in one or more images using the one or more correlation filters.
2 . The method of claim 1 , wherein the learning includes:
generating at least one modified image corresponding to the one or more image areas using the one or more occlusion maps to alter pixels in the one or more image areas that depict the one or more occluding regions; and
determining the one or more values of the one or more correlation filters using the at least one modified image.
3 . The method of claim 1 , wherein the one or more correlation filters are determined using image data corresponding to the one or more image areas, and the image data is generated based at least on one or more of:
masking, from the image data, the one or more occluding regions from the one or more image areas;
blurring the one or more occluding regions in the one or more image areas; or
removing the one or more occluding regions from the one or more image areas.
4 . The method of claim 1 , wherein the generating the one or more occlusion maps includes applying the one or more image areas to one or more machine learning models to predict a background corresponding to the one or more objects.
5 . The method of claim 1 , wherein prior to the learning the one or more values, one or more second values of the one or more correlation filters were learned with respect to at least one unoccluded region of the one or more objects in at least one image area, and the one or more occlusion maps reduce or eliminate updated learning of the one or more second values from the one or more occluding regions.
6 . The method of claim 1 , wherein the learning is further from at least a portion of a background of the one or more objects.
7 . The method of claim 1 , wherein the one or more occlusion maps are generated using one or more machine learning models, one or more parameters of the one or more machine learning models are updated using a target object, and the one or more correlation filters are to detect the target object.
8 . The method of claim 1 , wherein the one or more image areas are determined based at least on search regions of one or more videos, the search regions identified based at least on tracking the one or more objects using versions of the one or more correlation filters, and wherein the determining comprises updating one or more of the versions of the one or more correlation filters.
9 . A system comprising:
one or more processors to perform operations including:
determining one or more locations of one or more objects in one or more images that depict one or more occluding regions that correspond to one or more occluded portions of the one or more objects and one or more unoccluded regions of the one or more objects;
based at least on the determining, identifying, in the one or more images, the one or more occluding regions of the one or more objects;
learning one or more values of one or more correlation filters from the one or more unoccluded regions of the one or more images based at least on reducing or eliminating the one or more occluding regions from the one or more images; and
detecting the one or more objects in at least one image using the one or more correlation filters.
10 . The system of claim 9 , wherein the learning includes:
generating at least one modified image corresponding to the one or more images using the one or more occluding regions; and
determining the one or more values for at least one target model of the one or more correlation filters using the at least one modified image.
11 . The system of claim 9 , wherein the one or more correlation filters are determined using image data corresponding to the one or more images, and the image data is generated based at least on one or more of:
masking, from the image data, one or more regions corresponding to the one or more occluding regions in the one or more images;
blurring the one or more regions; or
excluding the one or more regions from the image data.
12 . The system of claim 9 , wherein the identifying the one or more occluding regions includes applying the one or more images to one or more machine learning models to predict a background corresponding to the one or more objects.
13 . The system of claim 9 , wherein the identifying the one or more occluding regions includes applying the one or more images to one or more machine learning models to detect one or more foreground regions.
14 . The system of claim 9 , wherein the one or more occluding regions indicate one or more partial occlusions of the one or more objects in the one or more images.
15 . The system of claim 9 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for presenting at least one of virtual reality content or augmented reality content;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
16 . At least one processor comprising:
one or more circuits to detect one or more objects in one or more images using one or more correlation filters, one or more values of the one or more correlation filters learned, at least in part, from one or more unoccluded regions of at least one image that depicts the one or more unoccluded regions of the one or more objects and one or more occluding regions that correspond to one or more occluded portions of the one or more objects, the learning being based at least on reducing or eliminating, using one or more occlusions maps that identify the one or more occluding regions in the at least one image, the one or more occluding regions from the at least one image.
17 . The at least one processor of claim 16 , wherein the one or more values are learned from at least one modified image corresponding to the one or more images and generated using the one or more occlusion maps to determine the one or more values of the one or more correlation filters.
18 . The at least one processor of claim 16 , wherein the one or more correlation filters are determined based at least on using the one or more occlusions maps to perform one or more of masking one or more regions depicting the one or more occlusions in the one or more images, blurring the one or more regions, or excluding the one or more regions from image data.
19 . The at least one processor of claim 16 , wherein the one or more occlusion maps are generated using one or more machine learning models to predict at least one of a background corresponding to the one or more objects or one or more foreground regions.
20 . The at least one processor of claim 16 , wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for presenting at least one of virtual reality content or augmented reality content;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.