Spatio-Temporal Multi-Task Learning Transformer for Joint Moving Object Detection and Segmentation

Moving objects have special importance for Autonomous Driving tasks.\nDetecting moving objects can be posed as Moving Object Segmentation, by\nsegmenting the object pixels, or Moving Object Detection, by generating a\nbounding box for the moving targets. In this paper, we present a Multi-Task\nLearning architecture, based on Transformers, to jointly perform both tasks\nthrough one network. Due to the importance of the motion features to the task,\nthe whole setup is based on a Spatio-Temporal aggregation. We evaluate the\nperformance of the individual tasks architecture versus the MTL setup, both\nwith early shared encoders, and late shared encoder-decoder transformers. For\nthe latter, we present a novel joint tasks query decoder transformer, that\nenables us to have tasks dedicated heads out of the shared model. To evaluate\nour approach, we use the KITTI MOD [29] data set. Results show1.5% mAP\nimprovement for Moving Object Detection, and 2%IoU improvement for Moving\nObject Segmentation, over the individual tasks networks.\n