Dynamic Spectral Guided Spatial Sparse Transformer for Hyperspectral Image Reconstruction

Junyang Wang, Xiang Yan, Hanlin Qin, Naveed Akhtar, Shuowen Yang, Ajmal Mian

Research output: Contribution to journalArticlepeer-review

Abstract

Hyperspectral image (HSI) reconstruction plays a crucial role in compressive spectral imaging with coded aperture snapshot spectrometry. Although HSI reconstruction has attracted much attention in recent years, it remains a challenging problem. Existing deep learning-based methods leverage all the spectral information to reconstruct the HSI images without considering the spectral redundancy of HSI images, leading to high computational costs. In this paper, we present an efficient method named Dynamic Spectral Guided Spatial Sparse Transformer (DGST). Specifically, DGST consists of three core modules: (a) Spectral Sparse Multi-Head Self-Attention Hybrid Spatial Feature Enhancement Module (SSHE), which employs a top-<italic>k</italic> spectral sparsity method to filter noise and redundant spectral information while extracting spectral information from HSI; (b) Spatial Information Compensation Module (SIC), which utilizes a multi-scale approach to extract spatial information and compensates for the spatial information neglected by SSHE; (c) Mask-Guided Spatial Sparse Multi-Head Self-Attention Hybrid Spectral Enhancement Module (MSSHE), which dynamically generates masks to guide the filtering of irrelevant regions, reducing computational costs while focusing on spatial information reconstruction. Our DGST improves the quality of HSI reconstruction by integrating spatial-spectral details and global information. Extensive experiments on public HSI reconstruction benchmark datasets demonstrate that our approach achieves state-of-the-art performance in end-to-end hyperspectral reconstruction. The superior performance of the proposed DGST is showcased on real and simulated hyperspectral imaging datasets. The source code is released at: <uri>https://github.com/WangJunYang2000/DGST</uri>.

Original languageEnglish
Pages (from-to)15494-15511
Number of pages18
JournalIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Volume17
Early online date10 Sept 2024
DOIs
Publication statusPublished - 2024

Fingerprint

Dive into the research topics of 'Dynamic Spectral Guided Spatial Sparse Transformer for Hyperspectral Image Reconstruction'. Together they form a unique fingerprint.

Cite this