Exploring the Collaboration Between Convolutional Neural Networks and Transformers in Hyperspectral Image Classification

Abstract

Recently, Hyperspectral Image (HSIs) classification has become the research hotspot and gained widespread popularity from scholars worldwide. Convolutional neural networks (CNNs), owing to their powerful representation ability, have gone into the mainstream in HSIs classification tasks. However, limited by their inherent network backbones, CNNs exist deficiencies when representing the sequence attributes of spectral signatures. To solve this issue, transformers are the prior choice for performing outstandingly in solving the problem of middle and long-term dependencies in spectral dimension. However, Pure transformers’ backbones may lose the power to capture contextual or semantic characteristics locally. As a result, we are motivated to explore the collaboration between CNNs and transformers and design a novel collaborated network Conv-Former. Experimental results on three HSIs benchmark datasets demonstrate that the proposed model outperforms other comparison methods and achieves the promising classification performances.