Multimodal Sentiment Analysis aims to reason and fuse complementary affective cues from different modalities to recognize human emotions, among which text is widely regarded as the core foundation and plays a major role due to its ability to directly carry emotional semantics. However, text modality faces two inherent challenges: label confusion, where subtle differences between similar emotion labels easily lead to misclassification, and semantic ambiguity, where vague or conflicting utterances hinder accurate emotion judgment. Existing methods treat all samples equally during training, ignoring the learning difficulty of these complex samples. To address this, we propose a Text-driven Hybrid Curriculum Learning (THCL) framework that progressively trains the model from easy to difficult samples. Specifically, a label-level curriculum is designed to measure similarity by the geometric distance of emotion labels in the Valence-Arousal-Dominance (VAD) space, where smaller distances imply higher confusion; an utterance-level curriculum is put in to model internal semantic consistency using a game-theoretic attribution tree, where greater ambiguity indicates higher difficulty. A comprehensive difficulty score is defined to dynamically adjust sample order, enabling gradual adaptation to complex expressions. Experiments on CMU-MOSI and CMU-MOSEI datasets show that THCL consistently outperforms state-of-the-art baselines across multiple metrics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Text-Driven Hybrid Curriculum Learning for Multimodal Sentiment Analysis

  • Feifei Xu,
  • Puzhe Li,
  • Dongyang Li,
  • Luobin Huang,
  • Wenjing Zhu

摘要

Multimodal Sentiment Analysis aims to reason and fuse complementary affective cues from different modalities to recognize human emotions, among which text is widely regarded as the core foundation and plays a major role due to its ability to directly carry emotional semantics. However, text modality faces two inherent challenges: label confusion, where subtle differences between similar emotion labels easily lead to misclassification, and semantic ambiguity, where vague or conflicting utterances hinder accurate emotion judgment. Existing methods treat all samples equally during training, ignoring the learning difficulty of these complex samples. To address this, we propose a Text-driven Hybrid Curriculum Learning (THCL) framework that progressively trains the model from easy to difficult samples. Specifically, a label-level curriculum is designed to measure similarity by the geometric distance of emotion labels in the Valence-Arousal-Dominance (VAD) space, where smaller distances imply higher confusion; an utterance-level curriculum is put in to model internal semantic consistency using a game-theoretic attribution tree, where greater ambiguity indicates higher difficulty. A comprehensive difficulty score is defined to dynamically adjust sample order, enabling gradual adaptation to complex expressions. Experiments on CMU-MOSI and CMU-MOSEI datasets show that THCL consistently outperforms state-of-the-art baselines across multiple metrics.