A Multimodal Sentiment Analysis Model Based on EEG and Text: BC-BiLSTM
摘要
Sentiment analysis is a key task in natural language processing for interpreting human emotions. To address the limitations of unimodal approaches, this study proposes a deep learning-based framework that integrates electroencephalogram (EEG) signals and textual data for sentiment classification. The EEG modality is encoded using a 1D convolutional neural network (CNN) to extract discriminative frequency band energy features, while the text modality is represented using contextual embeddings from a pretrained Bidirectional Encoder Representations from Transformers (BERT) model. The two modalities are fused and passed through a bidirectional long short-term memory (BiLSTM) with an attention mechanism to capture temporal and semantic dependencies. Experimental results on the Zurich Cognitive Language Processing (ZuCo) dataset show that the proposed multimodal model significantly outperforms both EEG-only and text-only baselines, achieving an accuracy of 87.29%, with macro-averaged precision, recall, and F1 score of 87.39%, 87.25%, and 87.25%, respectively. Compared with several state-of-the-art multimodal models, our method also demonstrates superior performance. These findings confirm the effectiveness of EEG-text fusion in capturing complementary information and improving sentiment analysis performance.