Joint object tracking

1 . A method comprising:

determining, based at least on a first two dimensional (2D) detected location associated with a tracked object at a first time as determined using first image data, a 2D estimated location associated with the tracked object at a second time;

determining, based at least on a first three-dimensional (3D) detected location associated with the tracked object at the first time as determined using the first image data, a 3D estimated location associated with the tracked object at the second time;

determining, based at least on second image data associated with the second time, a second 2D detected location associated with a detected object and a second 3D detected location associated with the detected object;

determining, based at least on comparing the 2D estimated location to the second 2D detected location a first cost of associating the detected object with the tracked object;

determining, based at least on comparing the 3D estimated location to the second 3D detected location, a second cost of associating the detected object with the tracked object;

determining, based at least on a third cost of associating the detected object with the tracked object that is based at least on the first cost and the second cost, that the detected object corresponds to the tracked object; and

causing a machine to navigate within an environment based at least on the determining that the detected object corresponds to the tracked object.

2 . The method of claim 1 , further comprising:

determining, based at least on the first cost and the second cost, the third cost of associating the detected object with the tracked object.

3 . The method of claim 1 , wherein the determining the 2D estimated location associated with the tracked object comprises:

determining, based at least on the first image data and the first 2D detected location, one or more first feature points associated with a first image represented by the first image data;

determining, based at least on the second image data, one or more second feature points associated with a second image represented by the second image data;

determining, based at least on the one or more first feature points and the one or more second feature points, at least one of a scalar change or a translation; and

determining, based at least on the first 2D detected location and the at least one of the scalar change or the translation, the 2D estimated location associated with the tracked object.

4 . The method of claim 1 , wherein the determining the 3D estimated location associated with the tracked object is further based at least on one or more of a velocity associated with the tracked object, an acceleration associated with the tracked object, or an orientation associated with the tracked object.

5 . The method of claim 1 , further comprising at least one of:

generating, based at least on the detected object corresponding to the tracked object, a new track associated with the tracked object, the new track being associated with at least one of the second 2D detected location or the second 3D detected location; or

updating, based at least on the detected object corresponding to the tracked object, an existing track associated with the tracked object, the updating including associating the existing track with at least one of the second 2D detected location or the second 3D detected location.

6 . The method of claim 1 , further comprising:

determining, based at least on third image data generated subsequent the second image data, a third 2D detected location associated with the tracked object;

determining, based at least on the third 2D detected location associated with the tracked object, a second 2D estimated location associated with the tracked object; and

determining, based at least on fourth image data generated at substantially a same time as the second image data, a fourth 2D detected location associated with the detected object,

wherein the determining that the detected object corresponds to the tracked object is further based at least on the second 2D estimated location and the fourth 2D detected location.

7 . A system comprising:

one or more processors to:

determine, based at least on first image data associated with a first time, a first three-dimensional (3D) detected location associated with a tracked object and a first two-dimensional (2D) detected location associated with the tracked object;

determine, based at least on second image data associated with a second time, a second 3D detected location associated with a detected object and a second 2D detected location associated with the detected object;

determine, based at least on comparing the second 3D detected location to a 3D estimated location associated with the tracked object at the second time as determined using the first 3D detected location, a first cost of associating the detected object with the tracked object;

determine, based at least on comparing the second 2D detected location to a 2D estimated location associated with the tracked object at the second time as determined using the first 2D detected location, a second cost of associating the detected object with the tracked object;

determine, based at least on a third cost of associating the detected object with the tracked object that is based at least on the first cost and the second cost, that the detected object corresponds to the tracked object; and

cause a machine to navigate within an environment based at least on the detected object being determined to correspond to the tracked object.

8 . The system of claim 7 , wherein the one or more processors are further to:

determine, based at least on the comparing the second 3D detected location to the 3D estimated location, one or more first differences between the 3D estimated location and the second 3D detection location, wherein the first cost is determined based at least on the one or more first differences; and

determined, based at least on the comparing the second 2D detected location to the 2D estimated location, one or more second differences between the 2D estimated location and the second 2D detected location, wherein the second cost is determined based at least on the one or more second differences.

9 . The system of claim 7 , wherein the 3D estimated location associated with the tracked object is further determined based at least on one or more of a velocity associated with the tracked object, an acceleration associated with the tracked object, or an orientation associated with the tracked object.

10 . The system of claim 7 , wherein the one or more processors are further to:

determine, based at least on the first image data and the first 2D detected location, one or more first feature points associated with a first image represented by the first image data;

determine, based at least on the second image data, one or more second feature points associated with a second image represented by the second image data;

determine, based at least on the one or more first feature points and the one or more second feature points, at least one of a scalar change or a translation; and

determine, based at least on the first 2D detected location and the at least one of the scalar change or the translation, the 2D estimated location associated with the tracked object.

11 . The system of claim 7 , wherein the one or more processors are further to update, based at least on the detected object corresponding to the tracked object, a track to be associated with at least one of the second 3D detected location or the second 2D detected location, the track being associated with the tracked object.

12 . The system of claim 7 , wherein the one or more processors are further to generate, based at least on the detected object corresponding to the tracked object, a track associated with the tracked object, the track being associated with at least one of the second 3D detected location or the second 2D detected location.

13 . The system of claim 7 , wherein the system is comprised in at least one of:

a control system for an autonomous or semi-autonomous machine;

a perception system for an autonomous or semi-autonomous machine;

a system for performing simulation operations;

a system for performing digital twin operations;

a system for performing light transport simulation;

a system for performing collaborative content creation for 3D assets;

a system for generating or presenting at least one of mixed reality content, virtual reality content, or augmented reality content;

a system for performing deep learning operations;

a system implemented using an edge device;

a system implemented using a robot;

a system for performing conversational AI operations;

a system for generating synthetic data;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

14 . The system of claim 7 , wherein the one or more processors are further to:

determine a fourth cost of associating the detected object with a second tracked object,

wherein the detected object is further determined to correspond to the tracked object based at least on the fourth cost.

15 . One or more processors comprising:

processing circuitry to:

determine, based at least on first image data associated with a first time, a first two-dimensional (2D) detected location associated with a tracked object and a first three-dimensional (3D) detected location associated with the tracked object;

determine, based at least on the first 2D detected location and the first 3D detected location, a 2D estimated location and a 3D estimated location associated with the tracked object at a future time;

determine, based at least on second image data associated with a second time, a second 2D detected location and a second 3D detected location associated with a detected object;

determine, based at least on comparing the 2D estimated location to the second 2D detected location, a first cost of associating the detected object with the tracked object;

determine, based at least on comparing the 3D estimated location to the second 3D detected location, a second cost of associating the detected object with the tracked object;

determine, based at least on a third cost of associating the detected object with the tracked object that is based at least on the first cost and the second cost, that the detected object corresponds to the tracked object; and

cause a machine to navigate within an environment based at least on the detected object being determined to correspond the tracked object.

16 . The one or more processors of claim 15 , wherein the one or more processors are comprising in at least one of:

a control system for an autonomous or semi-autonomous machine;

a perception system for an autonomous or semi-autonomous machine;

a system for performing simulation operations;

a system for performing digital twin operations;

a system for performing light transport simulation;

a system for performing collaborative content creation for 3D assets;

a system for generating or presenting at least one of mixed reality content, virtual reality content, or augmented reality content;

a system for performing deep learning operations;

a system implemented using an edge device;

a system implemented using a robot;

a system for performing conversational AI operations;

a system for generating synthetic data;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

17 . The one or more processors of claim 15 , wherein the processing circuitry is further to:

determine the third cost of associating the detected object with the tracked object based at least on the first cost and the second cost.

18 . The one or more processors of claim 15 , wherein the processing circuitry is further to generate, based at least on the detected object corresponding to the tracked object, a track associated with the tracked object, the track being associated with at least one of the second 2D detected location or the second 3D detected location.

19 . The one or more processors of claim 15 , wherein the processing circuitry is further to:

update, based at least on the detected object corresponding to the tracked object, a track associated with the tracked object,

wherein the machine is caused to navigate within the environment based at least on the track as updated.

20 . The one or more processors of claim 15 , wherein the processing circuitry is further to:

determine a fourth cost of associating the detected object with a second tracked object,

wherein the detected object is further determined to correspond to the tracked object based at least on the fourth cost.