Stereo-based 3D Anomaly Object Detection for Autonomous Driving: A New Dataset and Baseline

Three-dimensional detection technology is widely used in the field of autonomous driving, with its application scenarios gradually expanding from enclosed highways to open conventional roads. For anomaly categories that appear on the road, 3-D detection models trained on closed sets often misdetect. To address this risk, it is necessary to enhance the generalization ability of 3-D detection models for targets of arbitrary shapes and to possess the capability to filter out anomalies. The generalization of 3-D detection is limited by two factors: the coupled training of 2-D and 3-D, and the insufficient diversity in the scale distribution of training samples. This article proposes a stereo-based 3-D anomaly object detection (S3AD) algorithm, which decouples the training strategy of 3-D and 2-D to release the generalization ability for arbitrary 3-D foreground detection, and proposes an anomaly scoring algorithm based on foreground confidence prediction, achieving target-level anomaly scoring. In order to further verify and enhance the generalization of anomaly detection, we use a 3-D rendering method to synthesize two augmented reality binocular stereo 3-D detection datasets, which are named KITTI-AR. KITTI-AR extends upon KITTI by adding 97 new categories. The KITTI-AR-ExD subset includes 39 common categories as extra training data to address the sparse sample distribution issue. Additionally, 58 rare categories form the KITTI-AR-OoD subset, which are used to simulate zero-shot scenarios in real-world settings. Finally, the performance of the algorithm and the dataset is verified in the experiments. (Code and dataset can be obtained at https://github.com/shiyi-mu/S3AD-Code)