Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction

Instance segmentation of unknown objects from images is regarded as relevant\nfor several robot skills including grasping, tracking and object sorting.\nRecent results in computer vision have shown that large hand-labeled datasets\nenable high segmentation performance. To overcome the time-consuming process of\nmanually labeling data for new environments, we present a transfer learning\napproach for robots that learn to segment objects by interacting with their\nenvironment in a self-supervised manner. Our robot pushes unknown objects on a\ntable and uses information from optical flow to create training labels in the\nform of object masks. To achieve this, we fine-tune an existing DeepMask\nnetwork for instance segmentation on the self-labeled training data acquired by\nthe robot. We evaluate our trained network (SelfDeepMask) on a set of real\nimages showing challenging and cluttered scenes with novel objects. Here,\nSelfDeepMask outperforms the DeepMask network trained on the COCO dataset by\n9.5% in average precision. Furthermore, we combine our approach with recent\napproaches for training with noisy labels in order to better cope with induced\nlabel noise.\n