TransCenter: Transformers with Dense Representations for Multiple-Object Tracking

Transformers have proven superior performance for a wide variety of tasks\nsince they were introduced. In recent years, they have drawn attention from the\nvision community in tasks such as image classification and object detection.\nDespite this wave, an accurate and efficient multiple-object tracking (MOT)\nmethod based on transformers is yet to be designed. We argue that the direct\napplication of a transformer architecture with quadratic complexity and\ninsufficient noise-initialized sparse queries - is not optimal for MOT. We\npropose TransCenter, a transformer-based MOT architecture with dense\nrepresentations for accurately tracking all the objects while keeping a\nreasonable runtime. Methodologically, we propose the use of image-related dense\ndetection queries and efficient sparse tracking queries produced by our\ncarefully designed query learning networks (QLN). On one hand, the dense\nimage-related detection queries allow us to infer targets' locations globally\nand robustly through dense heatmap outputs. On the other hand, the set of\nsparse tracking queries efficiently interacts with image features in our\nTransCenter Decoder to associate object positions through time. As a result,\nTransCenter exhibits remarkable performance improvements and outperforms by a\nlarge margin the current state-of-the-art methods in two standard MOT\nbenchmarks with two tracking settings (public/private). TransCenter is also\nproven efficient and accurate by an extensive ablation study and comparisons to\nmore naive alternatives and concurrent works. For scientific interest, the code\nis made publicly available at https://github.com/yihongxu/transcenter.\n