Transformers and Foundation Models for Hyperspectral Imaging
摘要
Hyperspectral imaging (HSI) presents unique challenges for image interpretation due to its high spectral resolution and substantial sensor variability across datasets, often resulting in inconsistent band numbers even within data from the same sensor. This variability calls for models that can adaptively fine-tune while preserving robust pre-training feature representations. Addressing these challenges requires both architectural innovations to enhance adaptivity and the development of more diverse and comprehensive pre-training datasets. Although recent large-scale hyperspectral datasets have marked notable progress, hyperspectral data still lag behind RGB and multispectral data in terms of scale and quality. Expanding hyperspectral datasets, therefore, remains a critical research priority. Moreover, integrating hyperspectral interpretation into general-purpose foundation models is an emerging direction. Existing foundation models predominantly support RGB, video, and text modalities, with limited attention to HSI-specific characteristics such as spectral properties, imaging mechanisms, and geographic context. Future efforts should focus on effectively fusing hyperspectral data with these models to improve interpretability and generalization. Enhancing the interpretability of hyperspectral foundation models, in particular, is expected to become a vital research frontier.