<p>Aspect-based multimodal sentimental analysis (ABMSA) aims to identify sentiment polarities for aspects involved in multimodal data. Existing approaches exhibit limitations concerning the elimination of semantic overlap within unimodal data and the extraction of discriminative aspect-guided emotional features. Moreover, they predominantly depend on visual pre-trained models to generate high-level features, while overlooking low-level detail information associated with emotional expression. To alleviate the aforementioned issues, a novel Adaptive Graph interaction guided Correlation and Discriminant learning (AGCD) method is proposed for ABMSA. Specifically, a channel-based bidirectional attention fusion module is constructed to adaptively explore the complementary of cross-layer features in the deep convolutional networks, thereby generating visual features enriched with detailed and semantic information. Based on this, the adaptive prior learning of orthogonal-constrained intra-modality graph convolution is introduced to mitigate semantic overlap during intra-modal contextual dependency learning, and cross-modality bidirectional interaction mechanism is furthered designed to capture the inter-modality local semantic associations. Particularly, an aspect-oriented deep association discriminant learning mechanism is designed to enhance the tightness of paired multi-modality aspect representations while simultaneously converging multi-modality aspect representations of the same polarity and diverging those of different polarities. To verify the effectiveness of the proposed method, a large number of experiments are performed on two public multimedia Twitter datasets, and the experimental results show that our AGCD method outperforms the compared methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive graph interaction guided correlation and discriminant learning for aspect-based multimodal sentiment analysis

  • Shunjie Wang,
  • Guoyong Cai,
  • Guangrui Lv

摘要

Aspect-based multimodal sentimental analysis (ABMSA) aims to identify sentiment polarities for aspects involved in multimodal data. Existing approaches exhibit limitations concerning the elimination of semantic overlap within unimodal data and the extraction of discriminative aspect-guided emotional features. Moreover, they predominantly depend on visual pre-trained models to generate high-level features, while overlooking low-level detail information associated with emotional expression. To alleviate the aforementioned issues, a novel Adaptive Graph interaction guided Correlation and Discriminant learning (AGCD) method is proposed for ABMSA. Specifically, a channel-based bidirectional attention fusion module is constructed to adaptively explore the complementary of cross-layer features in the deep convolutional networks, thereby generating visual features enriched with detailed and semantic information. Based on this, the adaptive prior learning of orthogonal-constrained intra-modality graph convolution is introduced to mitigate semantic overlap during intra-modal contextual dependency learning, and cross-modality bidirectional interaction mechanism is furthered designed to capture the inter-modality local semantic associations. Particularly, an aspect-oriented deep association discriminant learning mechanism is designed to enhance the tightness of paired multi-modality aspect representations while simultaneously converging multi-modality aspect representations of the same polarity and diverging those of different polarities. To verify the effectiveness of the proposed method, a large number of experiments are performed on two public multimedia Twitter datasets, and the experimental results show that our AGCD method outperforms the compared methods.