Hyperspectral imaging (HSI) offers a non-invasive approach to intraoperative brain tumour detection; however, accurately delineating tumour margins remains a formidable challenge. This paper presents the Spatial-Spectral Transformer Network (S2T-Net), a novel deep learning (DL) framework that addresses highly accurate pixel-wise hyperspectral brain tissue classification. S2T-Net utilises an innovative Conv3D patch embedding mechanism and a post-encoder attention block leveraging a CLS token to pose spatial-spectral feature queries to improve the feature discrimination. When tested on an In Vivo HSI dataset, S2T-Net performed state-of-the-art, with 99.91% overall accuracy and 99.82% recall. Most importantly, it achieves a flawless precision of 100% for the tumour tissue class, which means zero false positives. These results make S2T-Net an extremely reliable and efficient system for real-time surgical guidance, far surpassing current approaches and enhancing the safety and efficacy of neurosurgery.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

S2T-Net: Spatial-Spectral Transformer Network for Hyperspectral Brain Tissue Classification

  • Raj Bahadur Singh,
  • Aloke Datta

摘要

Hyperspectral imaging (HSI) offers a non-invasive approach to intraoperative brain tumour detection; however, accurately delineating tumour margins remains a formidable challenge. This paper presents the Spatial-Spectral Transformer Network (S2T-Net), a novel deep learning (DL) framework that addresses highly accurate pixel-wise hyperspectral brain tissue classification. S2T-Net utilises an innovative Conv3D patch embedding mechanism and a post-encoder attention block leveraging a CLS token to pose spatial-spectral feature queries to improve the feature discrimination. When tested on an In Vivo HSI dataset, S2T-Net performed state-of-the-art, with 99.91% overall accuracy and 99.82% recall. Most importantly, it achieves a flawless precision of 100% for the tumour tissue class, which means zero false positives. These results make S2T-Net an extremely reliable and efficient system for real-time surgical guidance, far surpassing current approaches and enhancing the safety and efficacy of neurosurgery.