Multimodal aspect-based sentiment analysis (MABSA) aims to determine the sentiment polarity of aspect terms by jointly leveraging textual and visual modalities. However, existing methods face two critical challenges: (1) directly fusing heterogeneous pre-trained features often leads to modality misalignment, which hampers effective cross-modal interaction; and (2) associating visual content with specific aspect terms is difficult, particularly under semantic inconsistencies between modalities. To overcome these issues, we propose an end-to-end MABSA framework that integrates a BART-based pseudo-text generation strategy with an aspect-guided multi-path attention fusion mechanism. The former projects visual features into the textual semantic space to mitigate distributional gaps, while the latter directs attention toward pseudo-text regions that align with target aspects to enhance fine-grained semantic correspondence. The enriched pseudo-text, combined with original text and aspect representations, is fed into a pre-trained language model for sentiment prediction. Experimental results on the Twitter2015 and Twitter2017 datasets show that our approach consistently and significantly outperforms competitive baselines in terms of both accuracy and macro-F1 score.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Aspect-Based Sentiment Analysis via Aspect-Guided Pseudo-text Generation

  • Zhenhowe Liu,
  • Feng Yu,
  • Ru Xie,
  • Dequan Zheng

摘要

Multimodal aspect-based sentiment analysis (MABSA) aims to determine the sentiment polarity of aspect terms by jointly leveraging textual and visual modalities. However, existing methods face two critical challenges: (1) directly fusing heterogeneous pre-trained features often leads to modality misalignment, which hampers effective cross-modal interaction; and (2) associating visual content with specific aspect terms is difficult, particularly under semantic inconsistencies between modalities. To overcome these issues, we propose an end-to-end MABSA framework that integrates a BART-based pseudo-text generation strategy with an aspect-guided multi-path attention fusion mechanism. The former projects visual features into the textual semantic space to mitigate distributional gaps, while the latter directs attention toward pseudo-text regions that align with target aspects to enhance fine-grained semantic correspondence. The enriched pseudo-text, combined with original text and aspect representations, is fed into a pre-trained language model for sentiment prediction. Experimental results on the Twitter2015 and Twitter2017 datasets show that our approach consistently and significantly outperforms competitive baselines in terms of both accuracy and macro-F1 score.