Scholarlink

<p>Multimodal aspect-based sentiment analysis (MABSA) is an advanced technology to identify all aspect terms together with their respective sentiment mentioned in the multimodal data. Most existing MABSA methods encounter two main challenges: (1) The representations used in MABSA are directly encoded by the general pre-trained models, which are insensitive to identifying aspect-level sentiment. (2) Latent visual semantic information is underutilized when representing the key aspects and their sentimental polarities. To address the mentioned challenges, we propose an optimized multi-task learning framework using a tri-encoder with caption prompt (TECP), including MABSA and two auxiliary unimodal tasks to jointly learn the aspect-aware and sentiment-aware multimodal representation. In TECP, the merged-attention fusion network is designed to obtain multimodal features, which enhances the intra-modal and cross-modal semantic interactions. Within the tri-encoder, the caption encoder is designed to further generate visual caption feature as important clue to enrich the multimodal semantic and sentimental information. Moreover, within the caption encoder, the dependency weight attention network is proposed to focus on the aspect-level feature in the caption sentence. We conduct elaborate experiments and evaluate the performance of TECP with respect to Precision, Recall, and F1-score. Our TECP achieves SOTA results on two benchmark Twitter datasets in comparison with previous baseline models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-task learning framework using tri-encoder with caption prompt for multimodal aspect-based sentiment analysis

  • 期刊论文

  • 发表日期：

    2025年4月29日

查看全文
  • Yuanyuan Cai,
  • Fei Tong,
  • Qingchuan Zhang,
  • Haitao Xiong

摘要

Multimodal aspect-based sentiment analysis (MABSA) is an advanced technology to identify all aspect terms together with their respective sentiment mentioned in the multimodal data. Most existing MABSA methods encounter two main challenges: (1) The representations used in MABSA are directly encoded by the general pre-trained models, which are insensitive to identifying aspect-level sentiment. (2) Latent visual semantic information is underutilized when representing the key aspects and their sentimental polarities. To address the mentioned challenges, we propose an optimized multi-task learning framework using a tri-encoder with caption prompt (TECP), including MABSA and two auxiliary unimodal tasks to jointly learn the aspect-aware and sentiment-aware multimodal representation. In TECP, the merged-attention fusion network is designed to obtain multimodal features, which enhances the intra-modal and cross-modal semantic interactions. Within the tri-encoder, the caption encoder is designed to further generate visual caption feature as important clue to enrich the multimodal semantic and sentimental information. Moreover, within the caption encoder, the dependency weight attention network is proposed to focus on the aspect-level feature in the caption sentence. We conduct elaborate experiments and evaluate the performance of TECP with respect to Precision, Recall, and F1-score. Our TECP achieves SOTA results on two benchmark Twitter datasets in comparison with previous baseline models.