This study presents an attention-based transfer learning framework for integrating electroencephalography (EEG) and eye tracking data within Brain-Computer Interfaces (BCIs), aiming to enhance classification accuracy and reduce calibration time. Utilizing the MAMEM dataset, which includes synchronized EEG and eye tracking recordings from 34 participants, we developed a multi-modal deep learning architecture incorporating domain adaptation and attention-driven fusion mechanisms. Our preprocessing pipeline involved comprehensive signal cleaning, filtering, artifact removal, and synchronized feature extraction for both modalities. The proposed model achieved a classification accuracy of 87.6%, representing a 5.3% improvement over single-modality approaches, and reduced decision latency from 685 ms to 423 ms. Additionally, calibration time was decreased by approximately 86%, requiring only 42.8 s compared to the traditional 10–15 min. Transfer learning effectiveness was demonstrated through rapid adaptation, reaching 80% of maximum performance within roughly 43 s, and significantly reducing domain divergence. The model exhibited robustness under varying conditions, maintaining accuracy above 85% with noise levels up to 15 dB SNR and resilience to missing EEG channels up to 20%. Interpretability analyses highlighted the distinct contributions of neural and ocular features to decision-making processes, offering insights into feature importance dynamics. These findings underscore the framework’s potential for developing efficient, reliable, and user-adaptive BCIs suitable for real-world applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Attention-Based Transfer Learning for Multi-modal EEG and Eye Tracking in Brain-Computer Interfaces

  • Maryam Abbasi,
  • Sónia Brito-Costa,
  • Ana Rita Teixeira,
  • Pedro Martins

摘要

This study presents an attention-based transfer learning framework for integrating electroencephalography (EEG) and eye tracking data within Brain-Computer Interfaces (BCIs), aiming to enhance classification accuracy and reduce calibration time. Utilizing the MAMEM dataset, which includes synchronized EEG and eye tracking recordings from 34 participants, we developed a multi-modal deep learning architecture incorporating domain adaptation and attention-driven fusion mechanisms. Our preprocessing pipeline involved comprehensive signal cleaning, filtering, artifact removal, and synchronized feature extraction for both modalities. The proposed model achieved a classification accuracy of 87.6%, representing a 5.3% improvement over single-modality approaches, and reduced decision latency from 685 ms to 423 ms. Additionally, calibration time was decreased by approximately 86%, requiring only 42.8 s compared to the traditional 10–15 min. Transfer learning effectiveness was demonstrated through rapid adaptation, reaching 80% of maximum performance within roughly 43 s, and significantly reducing domain divergence. The model exhibited robustness under varying conditions, maintaining accuracy above 85% with noise levels up to 15 dB SNR and resilience to missing EEG channels up to 20%. Interpretability analyses highlighted the distinct contributions of neural and ocular features to decision-making processes, offering insights into feature importance dynamics. These findings underscore the framework’s potential for developing efficient, reliable, and user-adaptive BCIs suitable for real-world applications.