Recently, many transformer-based approaches have emerged in the field of hyperspectral image (HSI) classification. However, existing transformer-based works either model the information of spectral vectors using a transformer or introduce a vision transformer (ViT) for feature expression of the spatial patch after principal component analysis (PCA), neglecting the spectral-spatial correlation of HSI. Besides, local details and global distributions are always challenging to simultaneously extract in these methods. To address the above issues, a local-to-global transformer (LGFormer) is proposed in this paper. In detail, the proposed approach directly extracts inherent features on the originally spectral-spatial patch, which not only takes the spatial distribution and spectral continuity into account but also preserves the intrinsically spectral-spatial correlation of the HSI cube. Moreover, a local-to-global self-attention (LGSA) including 3-D convolutional neural network (CNN) and ViT is designed in the presented LGFormer. With the above HSI-tailored structure, both fine- and coarse-grained features can be captured by progressively spectral-spatial feature learning. Experimental results on benchmark HSI datasets demonstrate that the proposed LGFormer can outperform other methods in terms of higher accuracy and finer classification maps.
Paper
Full text
LGFormer: Local-to-Global Transformer for Hyperspectral Image Classification
Semantic Scholar · Environmental Science · 2023
Abstract
Recently, many transformer-based approaches have emerged in the field of hyperspectral image (HSI) classification. However, existing transformer-based works either model the information of spectral vectors using a transformer or introduce a vision transformer (ViT) for feature expression of the spatial patch after principal component analysis (PCA), neglecting the spectral-spatial correlation of HSI. Besides, local details and global distributions are always challenging to simultaneously extract in these methods. To address the above issues, a local-to-global transformer (LGFormer) is proposed in this paper. In detail, the proposed approach directly extracts inherent features on the originally spectral-spatial patch, which not only takes the spatial distribution and spectral continuity into account but also preserves the intrinsically spectral-spatial correlation of the HSI cube. Moreover, a local-to-global self-attention (LGSA) including 3-D convolutional neural network (CNN) and ViT is designed in the presented LGFormer. With the above HSI-tailored structure, both fine- and coarse-grained features can be captured by progressively spectral-spatial feature learning. Experimental results on benchmark HSI datasets demonstrate that the proposed LGFormer can outperform other methods in terms of higher accuracy and finer classification maps.