Joint Dynamic Electrode Permutation and Hierarchical Fusion for EEG-fNIRS Emotion Classification
摘要
Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are non-invasive neuro-imaging techniques that can be acquired synchronously. EEG captures electrophysiological activity, while fNIRS measures hemodynamic responses, providing complementary insights into neural processes. The integration of both modalities enhances emotion recognition, yet most existing methods rely on static electrode arrangements, which restrict the adaptability of functional brain networks across varying emotional states and limit effective bimodal fusion. To address these limitations, this paper proposes an Electrode Dynamic Permutation and Hierarchical Fusion (EDPHF) model for bimodal emotion recognition. EEG and fNIRS signals are modeled as graphs, with an end-to-end learnable dynamic adjacency matrix enabling adaptive electrode permutation. The model follows a three-stage cascade: extracting local spatial patterns via graph convolution and channel attention, capturing global dependencies with a Transformer, and adaptively fusing bimodal representations through modality attention. Experiments on a self-constructed dataset show that EDPHF outperforms state-of-the-art methods in recognition accuracy, validating its dynamic electrode permutation and hierarchical fusion strategies.