Object detection and property determination for autonomous vehicles
Systems, methods, tangible non-transitory computer-readable media, and devices for detecting objects are provided. For example, the disclosed technology can obtain a representation of sensor data associated with an environment surrounding a vehicle. Further, the sensor data can include sensor data points. A point classification and point property estimation can be determined for each of the sensor data points and a portion of the sensor data points can be clustered into an object instance based on the point classification and point property estimation for each of the sensor data points. A collection of point classifications and point property estimations can be determined for the portion of the sensor data points clustered into the object instance. Furthermore, object instance property estimations for the object instance can be determined based on the collection of point classifications and point property estimations for the portion of the sensor data points clustered into the object instance.
1. A computer-implemented method comprising:
obtaining training data comprising a first portion corresponding to a representation of sensor data comprising a plurality of sensor data points, and a second portion indicative of ground truth data, wherein the ground truth data is indicative of a labeled object instance property estimation that corresponds with an object instance associated with the first portion;
inputting, into a machine-learned detector model, the first portion of the training data,
wherein the machine-learned detector model is configured to determine point property estimations for at least a portion of the plurality of sensor data points represented in the first portion of the training data and, based on the point property estimations, determine a predicted object instance property estimation for the object instance;
receiving, as an output of the machine-learned detector model, a training output indicative of the predicted object instance property estimation;
determining a loss based on the training output, wherein the loss is based on a comparison of the predicted object instance property estimation to the labeled object instance property estimation of the ground truth data; and
modifying the machine-learned detector model based on the loss.
2. The computer-implemented method of claim 1 , wherein the machine-learned detector model is configured to generate the object instance based on the point property estimations.
3. The computer-implemented method of claim 1 , wherein modifying the machine-learned detector model based on the loss comprises modifying one or more weights associated with the machine-learned detector model.
4. The computer-implemented method of claim 1 , wherein modifying the machine-learned detector model comprises performing a backpropagation for the machine-learned detector model based on the loss.
5. The computer-implemented method of claim 1 , wherein the object instance associated with the first portion comprises an object instance for an object that was identified based on the sensor data.
6. The computer-implemented method of claim 1 , wherein the labeled object instance property estimation comprise at least a bounding shape corresponding to the object instance associated with the first portion.
7. The computer-implemented method of claim 6 , wherein the labeled object instance property estimations are at least (i) manually annotated or (ii) automatically annotated.
8. An autonomous vehicle control system comprising:
one or more processors; and
one or more tangible, non-transitory computer readable media that store instructions that are executable by the one or more processors to cause the autonomous vehicle control system to perform operations, the operations comprising:
obtaining training data comprising a first portion corresponding to a representation of sensor data comprising a plurality of sensor data points, and a second portion indicative of ground truth data, wherein the ground truth data is indicative of a labeled object instance property estimation that corresponds with an object instance associated with the first portion;
inputting, into a machine-learned detector model, the first portion of the training data,
wherein the machine-learned detector model is configured to determine point property estimations for at least a portion of the plurality of sensor data points represented in the first portion of the training data and, based on the point property estimations, determine a predicted object instance property estimation for the object instance;
receiving, as an output of the machine-learned detector model, a training output indicative of the predicted object instance property estimation;
determining a loss based on the training output, wherein the loss is based on a comparison of the predicted object instance property estimation to the labeled object instance property estimation of the ground truth data; and
modifying the machine-learned detector model based on the loss.
9. The autonomous vehicle control system of claim 8 , wherein the machine-learned detector model is configured to generate the object instance based on the point property estimations.
10. The autonomous vehicle control system of claim 8 , wherein modifying the machine-learned detector model based on the loss comprises modifying one or more weights associated with the machine-learned detector model.
11. The autonomous vehicle control system of claim 8 , wherein modifying the machine-learned detector model comprises performing a backpropagation for the machine-learned detector model based on the loss.
12. The autonomous vehicle control system of claim 8 , wherein the object instance associated with the first portion comprises an object instance for an object that was identified based on the sensor data.
13. The autonomous vehicle control system of claim 8 , wherein the labeled object instance property estimation comprise at least a bounding shape corresponding to the object instance associated with the first portion.
14. The autonomous vehicle control system of claim 13 , wherein the labeled object instance property estimations are at least (i) manually annotated or (ii) automatically annotated.
15. One or more tangible, non-transitory computer readable media that store instructions that are executable by one or more processors to cause the one or more processors to perform operations, the operations comprising:
obtaining training data comprising a first portion corresponding to a representation of sensor data comprising a plurality of sensor data points, and a second portion indicative of ground truth data, wherein the ground truth data is indicative of a labeled object instance property estimation that corresponds with an object instance associated with the first portion;
inputting, into a machine-learned detector model, the first portion of the training data,
wherein the machine-learned detector model is configured to determine point property estimations for at least a portion of the plurality of sensor data points represented in the first portion of the training data and, based on the point property estimations, determine a predicted object instance property estimation for the object instance;
receiving, as an output of the machine-learned detector model, a training output indicative of the predicted object instance property estimation;
determining a loss based on the training output, wherein the loss is based on a comparison of the predicted object instance property estimation to the labeled object instance property estimation of the ground truth data; and
modifying the machine-learned detector model based on the loss.
16. The one or more tangible, non-transitory computer readable media of claim 15 , wherein the machine-learned detector model is configured to generate the object instance based on the point property estimations.
17. The one or more tangible, non-transitory computer readable media of claim 15 , wherein modifying the machine-learned detector model based on the loss comprises modifying one or more weights associated with the machine-learned detector model.
18. The one or more tangible, non-transitory computer readable media of claim 15 , wherein modifying the machine-learned detector model comprises performing a backpropagation for the machine-learned detector model based on the loss.
19. The one or more tangible, non-transitory computer readable media of claim 15 , wherein the object instance associated with the first portion comprises an object instance for an object that was identified based on the sensor data.
20. The one or more tangible, non-transitory computer readable media of claim 15 , wherein the labeled object instance property estimation comprise at least a bounding shape corresponding to the object instance associated with the first portion.