Scholarlink

<p>Representation learning has shown significant improvement in Multimodal Sentiment Analysis (MSA). Existing studies, however, have not considered the multimodal heterogeneity within the sample and the diversity among samples, which may lead to a decrease in the model’s accuracy in real-world scenarios. In view of the aforementioned issues, we propose a multi-perspective decoupled representation learning method (MDR-MSA) for MSA. From three distinct perspectives, comprehensive multimodal sentiment features are extracted: diversity among samples, independence of samples and categories, and internal consistency. For the modal heterogeneity within the sample, modality-invariant and modality-specific subspace mapping models are developed to capture the consistency within the sample and the independence between samples and categories. A modal distance metric function and a multi-task learning approach are used to minimize the inherent gap between modalities, emphasizing the independence between samples and categories. For diversity among samples, a Res2Net-based Coordinate Attention module is utilized to emphasize the interaction among samples and cross-sample dependencies. Finally, a Transformer-based multi-head self-attention module is used to integrate feature representations from various modalities. Extensive experiments on two widely-used datasets demonstrate that the proposed method outperforms state-of-the-art models. Our code will be publicly available at <a href="https://github.com/lucky1104/MDR-MSA.">https://github.com/lucky1104/MDR-MSA.</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MDR-MSA: multi-perspective decoupled representation learning for multimodal sentiment analysis

  • 期刊论文

  • 发表日期：

    2025年12月4日

查看全文
  • Jiangying Du,
  • Yuxing Zhi,
  • Huaijun Wang,
  • Kan Wang,
  • Junhuai Li

摘要

Representation learning has shown significant improvement in Multimodal Sentiment Analysis (MSA). Existing studies, however, have not considered the multimodal heterogeneity within the sample and the diversity among samples, which may lead to a decrease in the model’s accuracy in real-world scenarios. In view of the aforementioned issues, we propose a multi-perspective decoupled representation learning method (MDR-MSA) for MSA. From three distinct perspectives, comprehensive multimodal sentiment features are extracted: diversity among samples, independence of samples and categories, and internal consistency. For the modal heterogeneity within the sample, modality-invariant and modality-specific subspace mapping models are developed to capture the consistency within the sample and the independence between samples and categories. A modal distance metric function and a multi-task learning approach are used to minimize the inherent gap between modalities, emphasizing the independence between samples and categories. For diversity among samples, a Res2Net-based Coordinate Attention module is utilized to emphasize the interaction among samples and cross-sample dependencies. Finally, a Transformer-based multi-head self-attention module is used to integrate feature representations from various modalities. Extensive experiments on two widely-used datasets demonstrate that the proposed method outperforms state-of-the-art models. Our code will be publicly available at https://github.com/lucky1104/MDR-MSA.