Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning

In this work, we introduce panoramic panoptic segmentation as the most\nholistic scene understanding both in terms of field of view and image level\nunderstanding for standard camera based input. A complete surrounding\nunderstanding provides a maximum of information to the agent, which is\nessential for any intelligent vehicle in order to make informed decisions in a\nsafety-critical dynamic environment such as real-world traffic. In order to\novercome the lack of annotated panoramic images, we propose a framework which\nallows model training on standard pinhole images and transfers the learned\nfeatures to a different domain. Using our proposed method, we manage to achieve\nsignificant improvements of over 5% measured in PQ over non-adapted models on\nour Wild Panoramic Panoptic Segmentation (WildPPS) dataset. We show that our\nproposed Panoramic Robust Feature (PRF) framework is not only suitable to\nimprove performance on panoramic images but can be beneficial whenever model\ntraining and deployment are executed on data taken from different\ndistributions. As an additional contribution, we publish WildPPS: The first\npanoramic panoptic image dataset to foster progress in surrounding perception.\n