Lighten-MST: Low-Light Spectral Reconstruction via Illumination-Guided Spectral-Aware Transformer

Abstract

Hyperspectral images (HSIs) offer a more nuanced depiction of scenes than traditional RGB images due to their superior spectral resolution and wide spectral bands. Due to the high cost of hyperspectral imaging systems, two widely used learning-based spectral reconstruction methods have been proposed: one reconstructing from measurements modulated by Coded Aperture Snapshot Spectral Imaging (CASSI), and the other reconstructing directly from RGB images. The mask encoding process in CASSI is particularly vulnerable to noise under low-light conditions, resulting in unstable reconstruction performance. Therefore, reconstructing HSIs from low-light RGB images is deemed a more dependable method to reconstruct HSIs with poor illumination. To address this challenge, we introduce a task called Low-light Spectral Reconstruction (LLSR) and propose the LOL-HSI dataset. Additionally, we enhance the MST++ network by creating an end-to-end architecture known as Lighten-MST, tailored for reconstructing HSIs from low-light RGB images. Our approach has achieved reconstruction performance exceeding 33 dB in PSNR on the LOL-HSI dataset, establishing a new state-of-the-art in LLSR. Our approach can enhance the robustness of spectral imaging systems in complex low-light conditions by utilizing simple RGB imaging systems to aid spectral imaging.