Improved Noise and Attack Robustness for Semantic Segmentation by Using Multi-Task Training with Self-Supervised Depth Estimation

While current approaches for neural network training often aim at improving\nperformance, less focus is put on training methods aiming at robustness towards\nvarying noise conditions or directed attacks by adversarial examples. In this\npaper, we propose to improve robustness by a multi-task training, which extends\nsupervised semantic segmentation by a self-supervised monocular depth\nestimation on unlabeled videos. This additional task is only performed during\ntraining to improve the semantic segmentation model's robustness at test time\nunder several input perturbations. Moreover, we even find that our joint\ntraining approach also improves the performance of the model on the original\n(supervised) semantic segmentation task. Our evaluation exhibits a particular\nnovelty in that it allows to mutually compare the effect of input noises and\nadversarial attacks on the robustness of the semantic segmentation. We show the\neffectiveness of our method on the Cityscapes dataset, where our multi-task\ntraining approach consistently outperforms the single-task semantic\nsegmentation baseline in terms of both robustness vs. noise and in terms of\nadversarial attacks, without the need for depth labels in training.\n