Robust Semantic Segmentation in Adverse Weather Conditions by means of Fast Video-Sequence Segmentation

Computer vision tasks such as semantic segmentation perform very well in good\nweather conditions, but if the weather turns bad, they have problems to achieve\nthis performance in these conditions. One possibility to obtain more robust and\nreliable results in adverse weather conditions is to use video-segmentation\napproaches instead of commonly used single-image segmentation methods.\nVideo-segmentation approaches capture temporal information of the previous\nvideo-frames in addition to current image information, and hence, they are more\nrobust against disturbances, especially if they occur in only a few frames of\nthe video-sequence. However, video-segmentation approaches, which are often\nbased on recurrent neural networks, cannot be applied in real-time applications\nanymore, since their recurrent structures in the network are computational\nexpensive. For instance, the inference time of the LSTM-ICNet, in which\nrecurrent units are placed at proper positions in the single-segmentation\napproach ICNet, increases up to 61 percent compared to the basic ICNet. Hence,\nin this work, the LSTM-ICNet is sped up by modifying the recurrent units of the\nnetwork so that it becomes real-time capable again. Experiments on different\ndatasets and various weather conditions show that the inference time can be\ndecreased by about 23 percent by these modifications, while they achieve\nsimilar performance than the LSTM-ICNet and outperform the single-segmentation\napproach enormously in adverse weather conditions.\n