EGMP: Enhanced Generative Multimodal Prompt for Few-Shot Multimodal Aspect-Based Sentiment Analysis
摘要
Multimodal Aspect-Based Sentiment Analysis (MABSA) aims to jointly identify aspect terms and their corresponding sentiment polarity from text-image pairs. Recent studies have applied prompt-based learning to MABSA tasks, which leverage the existing knowledge of pre-trained language models to fine-tuning with only a small number of samples, thereby significantly reducing the demand for data. However, these existing studies still exhibit notable deficiencies. They fail to fully explore the features of each modality and have only established a coarse-grained alignment between the textual and visual modalities, lacking a fine-grained cross-modal interaction. To address the aforementioned limitations, this paper proposes a novel Enhanced Generative Multimodal Prompt (EGMP) model. Specifically, we first employ GPT-4 combined with Chain-of-Thought (CoT) reasoning to generate image descriptions for visual semantic enhancement. Furthermore, to tackle the challenges of imprecise cross-modal alignment between image regions and textual aspects, a Syntax-aware Aspect-guided Fine-grained Alignment Module is introduced, which simultaneously filters semantically relevant information for each specific aspect from both textual tokens and visual patches, ultimately achieving aspect-region-opinion alignment. Finally, based on the semantically aligned representations, we construct enhanced multimodal embeddings with prompt for different tasks. Extensive experiments conducted on two benchmark datasets demonstrate that the proposed method outperforms the strong baseline model in two MABSA-related tasks.