Emotion recognition through integrated dual-channel EEG and peripheral physiological data
摘要
In recent years, there has been significant progress in emotion detection using physiological signals. Laboratory-based experiments typically use sophisticated systems like multiple channel Electroencephalograms (EEG), which offer higher accuracy but are impractical for real-world scenarios due to their complexity. In contrast, wearable devices for emotion detection, which measure peripheral physiological signals, offer more convenience but often lack the accuracy to decode complex emotions, particularly in inter-subject detection. This study aims to bridge this gap by exploring a practical solution by combining a two-channel EEG system (Fp1, Fp2) with peripheral signals such as Galvanic Skin Response (GSR) and Photoplethysmography (PPG) to enhance emotion classification accuracy. We utilize the publicly available Database for Emotion Analysis using Physiological Signals (DEAP) dataset to classify emotions and perform both unimodal and multimodal classification. For the multimodal approach, we fuse information from the three signal modalities using feature-level and decision-level fusion strategies. We performed inter-subject emotion classification using Leave-One-Subject-Out Cross-Validation (LOSO-CV) approach. The results obtained indicate that while the two-channel EEG alone surpasses GSR and PPG in performance, the feature-level fusion of all signals achieves the highest accuracy, with 74.24% for high vs low arousal and 74.78% for high vs low valence using a Random Forest (RF) classifier. These results are comparable to those from studies using full EEG setups for multimodal emotion detection. This study serves as a proxy for wearable devices in real-world applications, demonstrating the potential of simplified EEG setups combined with peripheral physiological signals for effective emotion detection.