Train-time loss in a system and method for calibrating object detection
A system and method of training a deep neural network for object detection in an object detection system. The object detection system including a camera and a controller including the DNN. The method including capturing an image by the camera, receiving the image, predicting, using the DNN, a bounding box and corresponding class label, evaluating the prediction with a total loss function including an object detection loss function, a box regression loss function, and a calibration loss function that takes into account precision and confidence. The method outputs a calibrated image with the object bounding box, the corresponding label, and a respective confidence score, in which the confidence score is a probability associated with the predicted class label.
1 . A method of using a pretrained deep neural network (DNN) for object detection in an object detection system, the object detection system including a camera and a controller including the pretrained DNN, the method comprising:
capturing an image by the camera;
receiving, by the controller, the image;
predicting, using the pretrained DNN, an object bounding box and a class label; and
outputting, by the controller, a calibrated image with the predicted object bounding box, the predicted class label, and a corresponding confidence score, the corresponding confidence score representing a confidence level associated with the predicted class label with respect to the predicted object bounding box,
wherein the pretrained DNN was trained based on a total loss function that includes an object detection loss function, a box regression loss function, and a train-time calibration loss function,
wherein the train-time calibration loss function is configured to maximize confidence scores for accurate training predictions and minimize confidence scores for inaccurate training predictions,
wherein a training prediction is accurate when both a predicted class label of the training prediction matches a ground truth class label, and an intersection-over-union (IoU) between a predicted object bounding box of the training prediction and a ground truth bounding box is greater than a threshold, and
wherein a training prediction is inaccurate when either a predicted class label of the training prediction does not match a ground truth class label, or an IoU between a predicted object bounding box of the training prediction and a ground truth bounding box is not greater than the threshold.
2 . The method of claim 1 , wherein the train-time calibration loss function is a function based on a number of detections, the method further comprising categorizing the detections as one of
accurate and confident,
accurate and not confident,
inaccurate and confident,
inaccurate and not confident.
3 . The method of claim 2 , further comprising modulating the confidence score of each categorized detection by a hyperbolic tangent function.
4 . The method of claim 1 , wherein the controller is in an autonomous vehicle, the method further comprising:
perceiving objects, by the pretrained DNN, with the camera in the autonomous vehicle.
5 . The method of claim 4 , wherein the autonomous vehicle includes a braking system, the method further comprising actuating the braking system based on an object detected by the object detection system.
6 . The method of claim 5 , wherein the autonomous vehicle includes a steering system, the method further comprising actuating the steering system in conjunction with the braking system based on the object detected by the object detection system.
7 . The method of claim 6 , wherein the autonomous vehicle includes a transmission system, the method further comprising actuating the transmission system, the steering system, and the braking system based on the object detected by the object detection system.
8 . An autonomous vehicle control system, comprising:
at least one camera mounted on the autonomous vehicle for capturing an image of an environment outside of the autonomous vehicle;
a controller connected to the at least one camera, the controller being configured to:
receive the image captured by the at least one camera,
predict, using a pretrained deep learning network (DNN), an object bounding box and a class label, and
output a calibrated image with the predicted object bounding box, the predicted class label, and a corresponding confidence score, the corresponding confidence score representing a confidence level associated with the predicted class label with respect to the predicted object bounding box,
wherein the pretrained DNN was trained based on a total loss function that includes an object detection loss function, a box regression loss function, and a train-time calibration loss function,
wherein the train-time calibration loss function is configured to maximize confidence scores for accurate training predictions and minimize confidence scores for inaccurate training predictions,
wherein a training prediction is accurate when both a predicted class label of the training prediction matches a ground truth class label, and an intersection-over-union (IoU) between a predicted object bounding box of the training prediction and a ground truth bounding box is greater than a threshold, and
wherein a training prediction is inaccurate when either a predicted class label of the training prediction does not match a ground truth class label, or an IoU between a predicted object bounding box of the training prediction and a ground truth bounding box is not greater than the threshold.
9 . The system of claim 8 , wherein the train-time calibration loss function is a function based on a number of detections, the controller is further configured to categorize the detections as one of
accurate and confident,
accurate and not confident,
inaccurate and confident,
inaccurate and not confident.
10 . The system of claim 9 , the controller further configured to modulate the confidence score of each categorized detection by a hyperbolic tangent function.
11 . The system of claim 8 , wherein the controller is further configured to:
perceive objects, by the pretrained DNN, based on the image captured by the camera.
12 . The system of claim 11 , further comprising a braking system, the controller is further configured to actuate the braking system based on an object detected by the controller.
13 . The system of claim 12 , further comprising a steering system, the controller further configured to actuate the steering system in conjunction with the braking system based on the object detected by the controller.
14 . The system of claim 13 , further comprising a transmission system, the controller further configured to actuate the transmission system, the steering system, and the braking system based on the object detected by the controller.
15 . A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method comprising:
capturing an image by a camera;
receiving, by the computer, the image;
predicting, using a pretrained deep learning neural network DNN, an object bounding box and a class label; and
outputting, by the computer, a calibrated image with the predicted object bounding box, the predicted class label, and a corresponding confidence score, the corresponding confidence score representing a confidence level associated with the predicted class label with respect to the predicted object bounding box,
wherein the pretrained DNN was trained based on a total loss function that includes an object detection loss function, a box regression loss function, and a train-time calibration loss function,
wherein the train-time calibration loss function is configured to maximize confidence scores for accurate training predictions and minimize confidence scores for inaccurate training predictions,
wherein a training prediction is accurate when both a predicted class label of the training prediction matches a ground truth class label, and an intersection-over-union (IoU) between a predicted object bounding box of the training prediction and a ground truth bounding box is greater than a threshold, and
wherein a training prediction is inaccurate when either a predicted class label of the training prediction does not match a ground truth class label, or an IoU between a predicted object bounding box of the training prediction and a ground truth bounding box is not greater than the threshold.
16 . The computer readable storage medium of claim 15 , wherein the train-time calibration loss function is a function based on a number of detections, the method further comprising categorizing the detections as one of
accurate and confident,
accurate and not confident,
inaccurate and confident,
inaccurate and not confident.
17 . The computer readable storage medium of claim 16 , the method further comprising modulating the confidence score of each categorized detection by a hyperbolic tangent function.