Object trajectory association and tracking

1. A computer-implemented method for generating a trajectory for an object, comprising:

obtaining, by a computing system comprising one or more computing devices, input data indicative of a respective trajectory for each of one or more first objects at a first time step;

obtaining, by the computing system, input data indicative of a respective trajectory for each of one or more second objects at a second time step, the second time step subsequent to the first time step and the one or more second objects comprising at least one of the one or more first objects;

generating, by the computing system using a machine-learned model, a temporally-consistent trajectory for at least one of the one or more first objects or the one or more second objects based at least in part on the input data indicative of the respective trajectory for the one or more first objects and the input data indicative of the respective trajectory for the one or more second objects; and

determining, by the computing system, a third predicted trajectory for the at least one of the one or more first objects or the one or more second objects for at least the second time step based at least in part on the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects;

wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises:

generating, by the machine-learned model, association data comprising data descriptive of an association of each respective trajectory of a first object of the one or more first objects to each respective trajectory of a second object of the one or more second objects; and

generating, by the machine-learned model, the temporally-consistent trajectory for at least one of the one or more second objects based at least in part on the association data.

2. The computer-implemented method of claim 1 , wherein the at least one of the one or more first objects or the one or more second objects comprises an occluded object; and

wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises:

propagating a previous trajectory associated with the occluded object forward in time.

3. The computer-implemented method of claim 1 , wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises:

generating, by the machine-learned model, association data by the machine-learned model, the association data comprising data descriptive of an association of each respective trajectory for the one or more first objects to each respective trajectory for the one or more second objects; and

generating, by the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more second objects based at least in part on the association data.

4. The computer-implemented method of claim 1 , wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises refining, by the machine-learned model, the respective trajectory for the at least one of the one or more first objects or the one or more second objects.

5. The computer-implemented method of claim 1 , wherein the at least one object comprises an object of the one or more second objects that is not highly associated with an object in the one or more first objects; and

wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises:

determining, by the computing system, to birth the temporally-consistent trajectory for the at least one of the one or more second objects.

6. The computer-implemented method of claim 1 , wherein the at least one object comprises an object of the one or more first objects that is not highly associated with an object in the one or more second objects; and

wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises:

determining, by the computing system, to reap the temporally-consistent trajectory for the at least one of the one or more second objects.

7. The computer-implemented method of claim 1 , wherein the input data indicative of the first predicted trajectory or the input data indicative of the second predicted trajectory comprises data output from a first stage model configured to perceive the one or more first objects or the one or more second objects and predict a trajectory for the one or more first objects or the one or more second objects based at least in part on sensor data obtained by one or more sensors of an autonomous vehicle.

8. The computer-implemented method of claim 1 , wherein the input data indicative of the first predicted trajectory or the input data indicative of the second predicted trajectory comprises one or more trajectories, bounding boxes, probabilities, feature maps, identifiers, and/or tracked or untracked higher order states for the one or more first objects or one or more second objects.

9. The computer-implemented method of claim 1 , wherein the first time step comprises a past time step and wherein the second time step comprises a current time step.

10. The computer-implemented method of claim 1 , wherein the machine-learned model has been trained using a loss function comprising an association loss parameter and one or more trajectory loss parameters.

11. The computer-implemented method of claim 1 , wherein the machine-learned model has been trained for one or more training steps using one or more ground-truth temporally-consistent trajectories and trained for one or more training steps using one or more outputs from a first stage model configured to determine a predicted trajectory for one or more objects using sensor data.

12. The computer-implemented method of claim 1 , wherein the machine-learned model comprises a LSTM, GRU or RNN model.

13. An autonomous vehicle, comprising:

one or more processors;

a memory comprising one or more tangible non-transitory computer-readable media, the media storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:

obtaining input data indicative of a respective trajectory for each of one or more first objects for a first time step from a first stage model configured to generate predicted trajectories for detected objects using sensor data from sensors of the autonomous vehicle obtained at the first time step;

obtaining input data indicative of a respective trajectory for each of one or more second objects for a second time step from the first stage model using sensor data from the sensors of the autonomous vehicle obtained at the second time step, the second time step subsequent to the first time step;

generating a temporally-consistent trajectory for at least one of the one or more second objects based at least in part on the input data indicative of the respective trajectory for the one or more first objects and the input data indicative of the respective trajectory for the one or more second objects using a machine-learned model; and

determining a third predicted trajectory for at least one of the one or more first objects or the one or more second objects for at least the second time step based at least in part on the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects;

wherein generating, by the computing system using the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more first objects or the one or more second objects comprises:

generating, by the machine-learned model, association data comprising data descriptive of an association of each respective trajectory of a first object of the one or more first objects to each respective trajectory of a second object of the one or more second objects; and

generating, by the machine-learned model, the temporally-consistent trajectory for the at least one of the one or more second objects based at least in part on the association data.

14. The autonomous vehicle of claim 13 , wherein the operations further comprise:

controlling the autonomous vehicle based at least in part on the third predicted trajectory for the at least one of the one or more first objects or one or more second objects.

15. The autonomous vehicle of claim 13 , further comprising:

one or more sensors configured to obtain sensor data;

wherein the first stage model is configured to perceive the one or more first objects or the one or more second objects and predict a trajectory for the one or more first objects or the one or more second objects based at least in part on the sensor data obtained by the one or more sensors; and

wherein the input data indicative of the first predicted trajectory or the input data indicative of the second predicted trajectory comprises data output from the first stage model.