White-box temperature scaling for uncertainty estimation in object detection

1 . A system, comprising a computer including a processor and a memory, the memory storing instructions executable by the processor programmed to:

determine uncertainty estimation in an object detection deep neural network (DNN) by:

retrieving a calibration dataset from a validation dataset, wherein the calibration dataset includes confidence scores of respective object classes in an image, including a background (BG) class, and includes logits and detection boxes prior to non-maximum suppression (pre-NMS);

determining an average number of pre-NMS detection boxes in non-BG classes as k;

extracting a top k pre-NMS detection boxes in the BG class using corresponding model scores;

determining background ground truth boxes in the calibration dataset by comparing ground truth boxes with detection boxes generated by the object detection DNN using an intersection over union (IoU) threshold;

correcting for class imbalance between the ground truth boxes and the background ground truth boxes in a ground truth class by updating the ground truth class to include a number of background ground truth boxes based on a number of ground truth boxes in the ground truth class; and

updating output datasets of the object detection DNN based on the class imbalance correction.

2 . The system of claim 1 , wherein the instructions for determining background ground truth boxes in the calibration set take both classification and regression branches into account by setting the IoU threshold to be the same as an IoU threshold used for training and evaluating the object detection DNN.

3 . The system of claim 1 , further including instructions to:

retrieve the calibration dataset by randomly choosing samples from an original pool of samples belonging to the BG class to match an average number of samples in non-BG classes.

4 . The system of claim 3 , further including instructions to:

report a final expected calibration error (ECE) as an average over 10 obtained calibration datasets.

5 . The system of claim 1 , wherein the instructions to estimate uncertainty further include instructions to:

determine a single scalar parameter of a temperature T for all classes by optimizing for a negative log likelihood (NLL) loss.

6 . The system of claim 5 , further including instructions to:

uniformly scale logit vectors of object detections in a test dataset prior to non-maximum suppression (pre-NMS) with the temperature T prior to a Sigmoid/Softmax layer.

7 . The system of claim 6 , further including instructions to:

after the Sigmoid/Softmax layer, perform non-maximum suppression on calibrated confidence scores with corresponding bounding box predictions to obtain final detections; and

actuate a vehicle component based upon an object detection determination of the object detection DNN.

8 . The system of claim 7 , wherein if an output Sigmoid class probability does not account for the BG class, further including instructions to:

explicitly add another dimension to the probability distribution to represent the BG class that is:

assigned a value of 0 for pre-NMS detection boxes with IoU greater than the IoU threshold and a value of 1 for pre-NMS detection boxes with IoU less than the IoU threshold; and

convert obtained confidence scores back to logits.

9 . A method comprising:

determining uncertainty estimation in an object detection deep neural network (DNN) by:

retrieving a calibration dataset from a validation dataset, wherein the calibration dataset includes confidence scores of respective object classes in an image, including a background (BG) class, and includes logits and detection boxes prior to non-maximum suppression (pre-NMS);

determining an average number of pre-NMS detection boxes in non-BG classes as k;

extracting a top k pre-NMS detection boxes in the BG class using corresponding model scores:

determining background ground truth boxes in the calibration dataset by comparing ground truth boxes with detection boxes generated by the object detection DNN using an intersection over union (IoU) threshold;

correcting for class imbalance between the ground truth boxes and the background ground truth boxes in a ground truth class by updating the ground truth class to include a number of background ground truth boxes based on a number of ground truth boxes in the ground truth class; and

updating output datasets of the object detection DNN based on the class imbalance correction.

10 . The method of claim 9 , wherein determining background ground truth boxes in the calibration dataset takes both classification and regression branches into account by setting the IoU threshold to be the same as an IoU threshold used for training and evaluating the object detection DNN.

11 . The method of claim 9 , further including:

obtaining the calibration dataset by randomly choosing samples from an original pool of samples belonging to the BG class to match an average number of samples in non-BG classes.

12 . The method of claim 11 , further including:

reporting a final expected calibration error (ECE) as an average over 10 obtained calibration datasets.

13 . The method of claim 9 , further including:

determining a single scalar parameter of a temperature T for all classes by optimizing for a negative log likelihood (NLL) loss.

14 . The method of claim 13 , further including:

uniformly scaling logit vectors of object detections in a test dataset prior to non-maximum suppression (pre-NMS) with the temperature T prior to a Sigmoid/Softmax layer.

15 . The method of claim 14 , further including:

after the Sigmoid/Softmax layer, performing non-maximum suppression on calibrated confidence scores with corresponding bounding box predictions to obtain final detections; and

actuating a vehicle component based upon an object detection determination of the object detection DNN.

16 . The method of claim 15 , wherein if an output Sigmoid class probability does not account for the BG class, further including explicitly adding another dimension to the probability distribution to represent the BG class that:

assigns a value of 0 for pre-NMS detection boxes with IoU greater than the IoU threshold and a value of 1 for pre-NMS detection boxes with IoU less than the IoU threshold; and

converts obtained confidence scores back to logits.