Object association for autonomous vehicles
Systems, methods, tangible non-transitory computer-readable media, and devices for associating objects are provided. For example, the disclosed technology can receive sensor data associated with the detection of objects over time. An association dataset can be generated and can include information associated with object detections of the objects at a most recent time interval and object tracks of the objects at time intervals in the past. A subset of the association dataset including the object detections that satisfy some association subset criteria can be determined. Association scores for the object detections in the subset of the association dataset can be determined. Further, the object detections can be associated with the object tracks based on the association scores for each of the object detections in the subset of the association dataset that satisfy some association criteria.
1. A computer-implemented method of object association,
the computer-implemented method comprising:
obtaining sensor data comprising a plurality of representations of one or more objects in an environment over a plurality of time intervals by one or more sensors of a vehicle, the plurality of representations comprising an object detection of the one or more objects at a most recent time interval of the plurality of time intervals and an object track of the one or more objects at a plurality of time intervals preceding the most recent time interval;
providing input data indicative of the plurality of representations of the one or more objects as an input to a machine-learned model;
receiving an output of the machine-learned model in response to receipt of the input data, wherein the output is indicative of a relationship between a respective object detection in the most recent time interval and a respective object track in the plurality of time intervals preceding the most recent time interval; and
associating the respective object detection with the respective object track in computer memory.
2. The computer-implemented method of claim 1 , wherein the output comprises an association classification indicative of whether to associate the respective object detection with the respective object track in the computer memory and an association score corresponding to the association classification, wherein associating the respective object detection with the respective object track in the computer memory comprises:
associating the respective object detection with the respective object track in the computer memory based at least in part on the association classification and the association score.
3. The computer-implemented method of claim 1 , wherein the input data indicative of the plurality of representations of the one or more objects comprises an association dataset, the association dataset comprising a plurality of combined feature vectors associated with the plurality of representations of the one or more objects in the environment.
4. The computer-implemented method of claim 3 , wherein the input data indicative of the plurality of representations of the one or more objects comprises a respective combined feature vector associated with the respective object detection and the respective object track.
5. The computer-implemented method of claim 1 , wherein the output comprises an association score indicative of whether to associate the respective object detection with the respective object track in the computer memory.
6. The computer-implemented method of claim 5 , wherein the association score is based at least in part on one or more features of the one or more objects, wherein the one or more features comprise at least one of a physical dimension, a velocity, an acceleration, or an orientation of at least one of the one or more objects.
7. The computer-implemented method of claim 5 , wherein associating the respective object detection with the respective object track in the computer memory comprises:
associating the respective object detection with the respective object track in the computer memory in response to the association score exceeding a predetermined association score threshold.
8. The computer-implemented method of claim 1 , wherein the output comprises an association classification indicative of whether to associate the respective object detection with the respective object track in the computer memory.
9. The computer-implemented method of claim 8 , wherein the association classification is based at least in part on whether the respective object detection and the respective object track achieve association subset criteria.
10. The computer-implemented method of claim 9 , wherein the association subset criteria is indicative of a distance between the respective object detection and the respective object track.
11. An autonomous vehicle control system, comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions that when executed by the one or more processors cause the autonomous vehicle control system to perform operations, the operations comprising:
obtaining sensor data comprising a plurality of representations of one or more objects in an environment over a plurality of time intervals by one or more sensors of a vehicle, the plurality of representations comprising an object detection of the one or more objects at a most recent time interval of the plurality of time intervals and an object track of the one or more objects at a plurality of time intervals preceding the most recent time interval;
providing input data indicative of the plurality of representations of the one or more objects as an input to a machine-learned model;
receiving an output of the machine-learned model in response to receipt of the input data, wherein the output is indicative of a relationship between a respective object detection in the most recent time interval and a respective object track in the plurality of time intervals preceding the most recent time interval; and
associating the respective object detection with the respective object track in computer memory.
12. The autonomous vehicle control system of claim 11 , wherein the sensor data comprises at least one of image data, radar data, or LiDAR data associated with the one or more objects.
13. The autonomous vehicle control system of claim 11 , wherein the input data indicative of the plurality of representations of the one or more objects comprises an association dataset, the association dataset comprising a plurality of combined feature vectors associated with the plurality of representations of the one or more objects in the environment.
14. The autonomous vehicle control system of claim 11 , wherein the input data indicative of the plurality of representations of the one or more objects comprises a respective combined feature vector associated with the respective object detection and the respective object track.
15. The autonomous vehicle control system of claim 14 , wherein the respective combined feature vector comprises one or more features for at least one of the respective object detection or the respective object track.
16. The autonomous vehicle control system of claim 15 , wherein the output comprises an association score indicative of whether to associate the respective object detection with the respective object track in the computer memory, wherein the association score is based at least in part on the one or more features for at least one of the respective object detection or the respective object track.
17. The autonomous vehicle control system of claim 15 , wherein the one or more features are indicative of at least one of a location, a physical dimension, a velocity, an acceleration, or an orientation for at least one of the respective object detection or the respective object track.
18. An autonomous vehicle comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that store instructions that when executed by the one or more processors cause the autonomous vehicle to perform operations, the operations comprising:
obtaining sensor data comprising a plurality of representations of one or more objects in an environment over a plurality of time intervals by one or more sensors of the autonomous vehicle, the plurality of representations comprising an object detection of the one or more objects at a most recent time interval of the plurality of time intervals and an object track of the one or more objects at a plurality of time intervals preceding the most recent time interval;
providing input data indicative of the plurality of representations of the one or more objects as an input to a machine-learned model;
receiving an output of the machine-learned model in response to receipt of the input data, wherein the output is indicative of a relationship between a respective object detection in the most recent time interval and a respective object track in the plurality of time intervals preceding the most recent time interval; and
associating the respective object detection with the respective object track in computer memory.
19. The autonomous vehicle of claim 18 , wherein the input data indicative of the plurality of representations of the one or more objects comprises an association dataset, the association dataset comprising a plurality of combined feature vectors associated with the plurality of representations of the one or more objects in the environment.
20. The autonomous vehicle of claim 18 , wherein the output comprises an association classification indicative of whether to associate the respective object detection with the respective object track in the computer memory and an association score corresponding to the association classification, wherein associating the respective object detection with the respective object track in computer memory comprises:
associating the respective object detection with the respective object track in computer memory based at least in part on the association classification and the association score.