Emotion recognition plays an important role in understanding human affective states and holds significant potential in diagnostic and therapeutic applications. This study presents a novel hybrid model that integrates convolutional neural networks and graph convolutional networks to improve multimodal emotion recognition by leveraging neuroimaging and physiological data. Specifically, functional magnetic resonance imaging, photoplethysmography, and respiratory signals were utilized to predict emotional valence. The proposed model processes fMRI data to capture spatial and temporal brain activity, while physiological signals are transformed into Gramian Angular Fields. to extract temporal features. The model’s performance was evaluated using a Leave-One-Subject-Out cross-validation strategy on the dataset provided by the ICBHI 2024 Scientific Challenge, achieving a competition score of 0.3314. The experimental results demonstrated an average accuracy of 70.42% ± 0.06% in the valence class prediction task and an average of the mean absolute error of 1.74 ± 0.46 in the valence rating prediction. Our hybrid model performance showed an accuracy of 83.33% in the classification task.