Cross-subject classification in brain-computer interface (BCI) systems using multimodal electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) data remains challenging because of subtle class boundaries and subject heterogeneity. To address these challenges, we introduced a novel simplified multimodal transformer network (SMTN). The SMTN leverages attention mechanisms to dynamically weigh input features, enhancing the focus on relevant regions while effectively minimizing noise and irrelevant data. This method captures intricate differences in brain activity patterns and manages data heterogeneity and class boundary issues by harnessing long-range dependencies and complex temporal dynamics. By integrating EEG and fNIRS data, the SMTN leverages the complementary strengths of both modalities, offering a robust framework for accurate cross-subject classification. Extensive experiments demonstrated SMTN’s superiority, achieving remarkable cross-subject accuracies of 97.3%, 96.3%, 98.1%, and 97.9% for the motor imagery, n-back, discrimination/selection response, and word generation tasks, respectively. These results underscore SMTN’s potential to revolutionize BCI applications and provide a powerful tool for enhanced cognitive and motor task analyses across diverse subjects. Additionally, SMTN’s adaptability to varying data quality and its potential for real-time applications highlight its practical significance in advancing neuroimaging research and BCI technology.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhanced Cross-Subject Classification of Hybrid EEG-fNIRS Data Using the Simplified Multimodal Transformer Network

  • Chayut Bunterngchit,
  • Jiaxing Wang,
  • Jianqiang Su,
  • Yihan Wang,
  • Shiqi Liu,
  • Zeng-Guang Hou

摘要

Cross-subject classification in brain-computer interface (BCI) systems using multimodal electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) data remains challenging because of subtle class boundaries and subject heterogeneity. To address these challenges, we introduced a novel simplified multimodal transformer network (SMTN). The SMTN leverages attention mechanisms to dynamically weigh input features, enhancing the focus on relevant regions while effectively minimizing noise and irrelevant data. This method captures intricate differences in brain activity patterns and manages data heterogeneity and class boundary issues by harnessing long-range dependencies and complex temporal dynamics. By integrating EEG and fNIRS data, the SMTN leverages the complementary strengths of both modalities, offering a robust framework for accurate cross-subject classification. Extensive experiments demonstrated SMTN’s superiority, achieving remarkable cross-subject accuracies of 97.3%, 96.3%, 98.1%, and 97.9% for the motor imagery, n-back, discrimination/selection response, and word generation tasks, respectively. These results underscore SMTN’s potential to revolutionize BCI applications and provide a powerful tool for enhanced cognitive and motor task analyses across diverse subjects. Additionally, SMTN’s adaptability to varying data quality and its potential for real-time applications highlight its practical significance in advancing neuroimaging research and BCI technology.