Adaptive graph interaction guided correlation and discriminant learning for aspect-based multimodal sentiment analysis
摘要
Aspect-based multimodal sentimental analysis (ABMSA) aims to identify sentiment polarities for aspects involved in multimodal data. Existing approaches exhibit limitations concerning the elimination of semantic overlap within unimodal data and the extraction of discriminative aspect-guided emotional features. Moreover, they predominantly depend on visual pre-trained models to generate high-level features, while overlooking low-level detail information associated with emotional expression. To alleviate the aforementioned issues, a novel Adaptive Graph interaction guided Correlation and Discriminant learning (AGCD) method is proposed for ABMSA. Specifically, a channel-based bidirectional attention fusion module is constructed to adaptively explore the complementary of cross-layer features in the deep convolutional networks, thereby generating visual features enriched with detailed and semantic information. Based on this, the adaptive prior learning of orthogonal-constrained intra-modality graph convolution is introduced to mitigate semantic overlap during intra-modal contextual dependency learning, and cross-modality bidirectional interaction mechanism is furthered designed to capture the inter-modality local semantic associations. Particularly, an aspect-oriented deep association discriminant learning mechanism is designed to enhance the tightness of paired multi-modality aspect representations while simultaneously converging multi-modality aspect representations of the same polarity and diverging those of different polarities. To verify the effectiveness of the proposed method, a large number of experiments are performed on two public multimedia Twitter datasets, and the experimental results show that our AGCD method outperforms the compared methods.