Learning Tracking Representations via Dual-Branch Fully Transformer Networks

We present a Siamese-like Dual-branch network based on solely Transformers\nfor tracking. Given a template and a search image, we divide them into\nnon-overlapping patches and extract a feature vector for each patch based on\nits matching results with others within an attention window. For each token, we\nestimate whether it contains the target object and the corresponding size. The\nadvantage of the approach is that the features are learned from matching, and\nultimately, for matching. So the features are aligned with the object tracking\ntask. The method achieves better or comparable results as the best-performing\nmethods which first use CNN to extract features and then use Transformer to\nfuse them. It outperforms the state-of-the-art methods on the GOT-10k and\nVOT2020 benchmarks. In addition, the method achieves real-time inference speed\n(about $40$ fps) on one GPU. The code and models will be released.\n