Multimodal Fusion Emotion Recognition with Cascade Forest Based on Electroencephalography and Facial Electromyography Signals
摘要
With the rapid development of human-computer interaction technology, emotion recognition has gradually become a key strategy that enables the interaction between machines and human emotions. Physiological signals serve as an important tool in emotion recognition. However, single-modal physiological signals tend to provide a limited amount of information and may disregard the complexity of emotional expression. Therefore, this paper proposes a multimodal fusion emotion recognition method based on electroencephalogram (EEG) and facial electromyogram (fEMG) for classifying three types of emotions (sadness, fear, and neutral). Differential entropy is extracted from five frequency bands of EEG signals, while autoregressive coefficients are extracted from fEMG signals to form the multimodal fusion features, and subsequently a cascade forest is constructed to complete the classification. An emotion dataset containing 32-channel EEG and 3-channel fEMG from 16 subjects was acquired, and the performance of the proposed method was evaluated on this dataset. The results show that multimodal fusion obtained an accuracy of 93.42%, which is significantly higher than single modality (EEG and facial EMG). The accuracy of the proposed method on all subjects’ data is over 80%, demonstrating the effectiveness of multimodal physiological signal fusion in emotion recognition.