From Black-box to White-box: Examining Confidence Calibration under different Conditions

Confidence calibration is a major concern when applying artificial neural\nnetworks in safety-critical applications. Since most research in this area has\nfocused on classification in the past, confidence calibration in the scope of\nobject detection has gained more attention only recently. Based on previous\nwork, we study the miscalibration of object detection models with respect to\nimage location and box scale. Our main contribution is to additionally consider\nthe impact of box selection methods like non-maximum suppression to\ncalibration. We investigate the default intrinsic calibration of object\ndetection models and how it is affected by these post-processing techniques.\nFor this purpose, we distinguish between black-box calibration with non-maximum\nsuppression and white-box calibration with raw network outputs. Our experiments\nreveal that post-processing highly affects confidence calibration. We show that\nnon-maximum suppression has the potential to degrade initially well-calibrated\npredictions, leading to overconfident and thus miscalibrated models.\n