Unsupervised Domain Adaptation with Multiple Domain Discriminators and Adaptive Self-Training

Unsupervised Domain Adaptation (UDA) aims at improving the generalization\ncapability of a model trained on a source domain to perform well on a target\ndomain for which no labeled data is available. In this paper, we consider the\nsemantic segmentation of urban scenes and we propose an approach to adapt a\ndeep neural network trained on synthetic data to real scenes addressing the\ndomain shift between the two different data distributions. We introduce a novel\nUDA framework where a standard supervised loss on labeled synthetic data is\nsupported by an adversarial module and a self-training strategy aiming at\naligning the two domain distributions. The adversarial module is driven by a\ncouple of fully convolutional discriminators dealing with different domains:\nthe first discriminates between ground truth and generated maps, while the\nsecond between segmentation maps coming from synthetic or real world data. The\nself-training module exploits the confidence estimated by the discriminators on\nunlabeled data to select the regions used to reinforce the learning process.\nFurthermore, the confidence is thresholded with an adaptive mechanism based on\nthe per-class overall confidence. Experimental results prove the effectiveness\nof the proposed strategy in adapting a segmentation network trained on\nsynthetic datasets like GTA5 and SYNTHIA, to real world datasets like\nCityscapes and Mapillary.\n