Weakly Supervised Few-shot Object Segmentation using Co-Attention with Visual and Semantic Embeddings

Significant progress has been made recently in developing few-shot object\nsegmentation methods. Learning is shown to be successful in few-shot\nsegmentation settings, using pixel-level, scribbles and bounding box\nsupervision. This paper takes another approach, i.e., only requiring\nimage-level label for few-shot object segmentation. We propose a novel\nmulti-modal interaction module for few-shot object segmentation that utilizes a\nco-attention mechanism using both visual and word embedding. Our model using\nimage-level labels achieves 4.8% improvement over previously proposed\nimage-level few-shot object segmentation. It also outperforms state-of-the-art\nmethods that use weak bounding box supervision on PASCAL-5i. Our results show\nthat few-shot segmentation benefits from utilizing word embeddings, and that we\nare able to perform few-shot segmentation using stacked joint visual semantic\nprocessing with weak image-level labels. We further propose a novel setup,\nTemporal Object Segmentation for Few-shot Learning (TOSFL) for videos. TOSFL\ncan be used on a variety of public video data such as Youtube-VOS, as\ndemonstrated in both instance-level and category-level TOSFL experiments.\n