Multi-object tracking using correlation filters in video analytics applications
In various examples, image areas may be extracted from a batch of one or more images and may be scaled, in batch, to one or more template sizes. Where the image areas include search regions used for localization of objects, the scaled search regions may be loaded into Graphics Processing Unit (GPU) memory and processed in parallel for localization. Similarly, where image areas are used for filter updates, the scaled image areas may be loaded into GPU memory and processed in parallel for filter updates. The image areas may be batched from any number of images and/or from any number of single- and/or multi-object trackers. Further aspects of the disclosure provide approaches for associating locations using correlation response values, for learning correlation filters in object tracking based at least on focused windowing, and for learning correlation filters in object tracking based at least on occlusion maps.
1. A computer-implemented method comprising:
identifying, from one or more first images of one or more videos, one or more image areas that correspond to one or more detected locations of one or more objects in the one or more videos;
based at least on the identifying, applying focused windowing to the one or more image areas, the focused windowing including blurring one or more regions corresponding to one or more backgrounds of the one or more objects in the one or more image areas based at least on a distance of the one or more regions from a target object in the one or more image areas;
updating, using the one or more image areas and based at least on the blurring, one or more correlation filters; and
generating, from the one or more correlation filters and based at least on the updating, one or more estimated object locations that correspond to one or more search regions of one or more second images of the one or more videos based at least on applying the one or more correlation filters to the one or more search regions.
2. The method of claim 1 , wherein the one or more image areas are based at least on first search regions of the one or more videos, the first search regions identified by tracking objects using versions of the one or more correlation filters, and wherein the updating comprises updating one or more of the versions of the one or more correlation filters.
3. The method of claim 1 , wherein the updating comprises initializing the one or more of the correlation filters based at least on determining one or more of a newly detected object or a newly tracked object.
4. The method of claim 1 , wherein the blurring is performed using a blur filter and one or more properties of an impulse response of the blur filter is adjusted based at least on a size of the target object.
5. The method of claim 1 , comprising generating occlusion maps from the one or more image areas, wherein each correlation filter of the correlation filters is generated from an image area of the one or more image areas using an occlusion map of the occlusion maps.
6. The method of claim 1 , further comprising:
extracting, from the one or more image areas, one or more feature channels of the one or more image areas, wherein updating additionally uses the one or more feature channels.
7. The method of claim 1 , further comprising:
determining a first estimated object location using a correlation response of a version of a correlation filter of the one or more correlation filters; and
determining a confidence score for a detected location of the one or more detected locations of one or more objects based at least on a correlation response value of the correlation response that corresponds to the detected location, wherein the determining the one or more correlation filters comprises updating the version of the correlation filter using a learning rate that is based at least on the confidence score.
8. A system comprising:
one or more processing units to perform operations including:
determining one or more image areas that correspond to one or more locations of one or more objects in one or more first images;
based at least on the determining, applying one or more focused windows to the one or more image areas, the applying including blurring one or more regions corresponding to one or more backgrounds of the one or more objects in the one or more image areas based at least on a distance of the one or more regions from a target object in the one or more image areas to generate one or more focused image areas;
learning one or more values of one or more correlation filters using the one or more focused image areas; and
generating one or more object locations based at least on applying the one or more correlation filters to one or more second images.
9. The system of claim 8 , wherein the blurring increases blur based at least on a distance from the target object in the one or more image areas.
10. The system of claim 8 , wherein the blurring includes one or more of:
blurring one or more color channels of the one or more image areas; or
blurring one or more feature channels of the one or more image areas.
11. The system of claim 8 , wherein the blurring uses a blur filter having one or more properties that are based at least on a size of the target object.
12. The system of claim 8 , wherein the one or more image areas include one or more scaled image areas that are scaled to a template size.
13. The system of claim 8 , wherein the operations comprise performing multi-object tracking across frames of one or more videos using the one or more object locations.
14. The system of claim 8 , wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for presenting at least one of virtual reality content or augmented reality content;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
15. A processor comprising:
one or more circuits to determine one or more object locations based at least on applying one or more correlation filters to one or more first images, the one or more correlation filters updated, at least in part, using one or more image areas based at least on applying focused windowing to one or more image areas, the focused windowing including blurring one or more regions corresponding to one or more backgrounds of one or more objects in the one or more image areas based at least on a distance of the one or more regions from a target object in the one or more image areas, the one or more image areas corresponding to one or more locations of the one or more objects in one or more second images.
16. The processor of claim 15 , wherein the blurring is performed using a Gaussian filter.
17. The processor of claim 15 , wherein the blurring includes one or more of:
blurring one or more color channels of the one or more image areas; or
blurring one or more feature channels of the one or more image areas.
18. The processor of claim 15 , further comprising determining one or more properties of a blur filter used to perform the blurring based at least on a size of the target object.
19. The processor of claim 15 , wherein the one or more image areas include one or more scaled image areas that are scaled to a template size.
20. The processor of claim 15 , wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for presenting at least one of virtual reality content or augmented reality content;
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.