Unsupervised Domain Adaptation through Inter-modal Rotation for RGB-D Object Recognition

Unsupervised Domain Adaptation (DA) exploits the supervision of a label-rich\nsource dataset to make predictions on an unlabeled target dataset by aligning\nthe two data distributions. In robotics, DA is used to take advantage of\nautomatically generated synthetic data, that come with "free" annotation, to\nmake effective predictions on real data. However, existing DA methods are not\ndesigned to cope with the multi-modal nature of RGB-D data, which are widely\nused in robotic vision. We propose a novel RGB-D DA method that reduces the\nsynthetic-to-real domain shift by exploiting the inter-modal relation between\nthe RGB and depth image. Our method consists of training a convolutional neural\nnetwork to solve, in addition to the main recognition task, the pretext task of\npredicting the relative rotation between the RGB and depth image. To evaluate\nour method and encourage further research in this area, we define two benchmark\ndatasets for object categorization and instance recognition. With extensive\nexperiments, we show the benefits of leveraging the inter-modal relations for\nRGB-D DA.\n