System and method for embedding uncertainty estimation into deep-neural-network-based autonomous driving perception frameworks
Embodiments of this disclosure can provide a system and method for training a perception model to perform an autonomous driving task. During operation, the system can obtain labeled training data comprising images captured by multiple cameras mounted at different locations on a vehicle, and the perception model can generate, in parallel, a prediction output associated with the task and a confidence score based on the labeled training data. The confidence score can indicate a level of uncertainty associated with the prediction output. The system can generate an uncertainty-weighted prediction based on ground truth indicated by the labeled training data, the prediction output, and the confidence score; compute a loss function based on the uncertainty-weighted prediction; and update the perception model based on the loss function.
1 . A method for training a perception model to perform an autonomous driving task, the method comprising:
obtaining labeled training data comprising images captured by multiple cameras mounted at different locations on a vehicle;
generating, in parallel by the perception model based on the labeled training data, a prediction output associated with the task and a confidence score, wherein the confidence score indicates a level of uncertainty correlated with visibility of the cameras and associated with the prediction output;
generating an uncertainty-weighted prediction based on ground truth indicated by the labeled training data, the prediction output, and the confidence score, wherein generating the uncertainty-weighted prediction comprises computing a weighted combination of the prediction output and the ground truth, and wherein the prediction output is scaled by the confidence score and the ground truth is scaled by complement of the confidence score;
computing a loss function based on the uncertainty-weighted prediction; and
updating the perception model based on the loss function.
2 . The method of claim 1 , wherein the autonomous driving task comprises:
a map vectorization task;
an object detection task;
a semantic segmentation task; or
a path prediction task.
3 . The method of claim 1 , wherein the perception model comprises a Bird's Eye View (BEV)-based perception model.
4 . The method of claim 1 , wherein the prediction output comprises one or more of a classification prediction and a regression prediction.
5 . The method of claim 1 , wherein computing the loss function further comprises adding a regularization loss term, and wherein the regularization loss term is determined based on the confidence score and a hyperparameter.
6 . The method of claim 5 , wherein the hyperparameter is dynamically adjusted based on a cap value of the regularization loss term.
7 . The method of claim 6 , further comprising:
decreasing the hyperparameter in response to the regularization loss term being greater than or equal to the cap value; and
increasing the hyperparameter in response to the regularization loss term being less than the cap value.
8 . The method of claim 1 , further comprising:
selecting a subset of the labeled training data; and
associating prediction outputs generated based on the selected subset of labeled training data with a static confidence score.
9 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for training a perception model to perform an autonomous driving task, the method comprising:
obtaining labeled training data comprising images captured by multiple cameras mounted at different locations on a vehicle;
generating, in parallel by the perception model based on the labeled training data, a prediction output associated with the task and a confidence score, wherein the confidence score indicates a level of uncertainty correlated with visibility of the cameras and associated with the prediction output;
generating an uncertainty-weighted prediction based on ground truth indicated by the labeled training data, the prediction output, and the confidence score, wherein generating the uncertainty-weighted prediction comprises computing a weighted combination of the prediction output and the ground truth, and wherein the prediction output is scaled by the confidence score and the ground truth is scaled by complement of the confidence score;
computing a loss function based on the uncertainty-weighted prediction; and
updating the perception model based on the loss function.
10 . The non-transitory computer-readable storage medium of claim 9 , wherein the perception model comprises a Bird's Eye View (BEV)-based perception model.
11 . The non-transitory computer-readable storage medium of claim 9 ,
wherein the prediction output comprises a classification prediction and/or a regression prediction.
12 . The non-transitory computer-readable storage medium of claim 9 , wherein computing the loss function further comprises adding a regularization loss term, and wherein the regularization loss term is determined based on the confidence score and a hyperparameter.
13 . The non-transitory computer-readable storage medium of claim 12 , wherein the hyperparameter is dynamically adjusted based on a cap value of the regularization loss term, and wherein the method further comprises:
decreasing the hyperparameter in response to the regularization loss term being greater than or equal to the cap value; and
increasing the hyperparameter in response to the regularization loss term being less than the cap value.
14 . The non-transitory computer-readable storage medium of claim 9 , wherein the method further comprises:
selecting a subset of the labeled training data; and
associating prediction outputs generated based on the selected subset of labeled training data with a static confidence score.
15 . A computing system, comprising:
a processor; and
a memory coupled to the processor and storing instructions that when executed by the processor cause the processor to perform a method for training a perception model to perform an autonomous driving task, the method comprising:
obtaining labeled training data comprising images captured by multiple cameras mounted at different locations on a vehicle;
generating, in parallel by the perception model based on the labeled training data, a prediction output associated with the task and a confidence score, wherein the confidence score indicates a level of uncertainty correlated with visibility of the cameras and associated with the prediction output;
generating an uncertainty-weighted prediction based on ground truth indicated by the labeled training data, the prediction output, and the confidence score, wherein generating the uncertainty-weighted prediction comprises computing a weighted combination of the prediction output and the ground truth, and wherein the prediction output is scaled by the confidence score and the ground truth is scaled by complement of the confidence score;
computing a loss function based on the uncertainty-weighted prediction; and
updating the perception model based on the loss function.
16 . The computing system of claim 15 , wherein computing the loss function further comprises adding a regularization loss term, and wherein the regularization loss term is determined based on the confidence score and a hyperparameter.
17 . The computing system of claim 16 , wherein the hyperparameter is dynamically adjusted based on a cap value of the regularization loss term, and wherein the method further comprises:
decreasing the hyperparameter in response to the regularization loss term being greater than or equal to the cap value; and
increasing the hyperparameter in response to the regularization loss term being less than the cap value.