Cross-Channel Dynamic Spatial–Spectral Fusion Transformer for Hyperspectral Image Classification

Abstract

Convolutional neural network (CNN) has achieved great success in hyperspectral image (HSI) classification. However, the local receptive field of CNNs leads to the drawback in extracting long-distance features. Transformer has excellent global modeling ability and shows good performance for HSI classification. The existing Transformer-based methods usually ignore the problem that the spatial information varies under different channels. To well describe the cross-channel dependencies, a cross-channel dynamic spatial–spectral fusion Transformer (CDSFT) is proposed in this article. In the proposed CDSFT, the multiscale and multichannel features are extracted and then cross-channel global features are extracted through transpose multihead self-attention (TMHSA). Next, a dynamic feature enhancement module and a spectral–spatial position attention (SSPA) module are designed to extract and enhance spectral–spatial joint features for classification. Experimental results on three well-known HSI datasets demonstrate the effectiveness of the proposed CDSFT method.