Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation

Video instance segmentation aims to detect, segment, and track objects in a\nvideo. Current approaches extend image-level segmentation algorithms to the\ntemporal domain. However, this results in temporally inconsistent masks. In\nthis work, we identify the mask quality due to temporal stability as a\nperformance bottleneck. Motivated by this, we propose a video instance\nsegmentation method that alleviates the problem due to missing detections.\nSince this cannot be solved simply using spatial information, we leverage\ntemporal context using inter-frame attentions. This allows our network to\nrefocus on missing objects using box predictions from the neighbouring frame,\nthereby overcoming missing detections. Our method significantly outperforms\nprevious state-of-the-art algorithms using the Mask R-CNN backbone, by\nachieving 36.0% mAP on the YouTube-VIS benchmark. Additionally, our method is\ncompletely online and requires no future frames. Our code is publicly available\nat https://github.com/anirudh-chakravarthy/ObjProp.\n