Scholarlink

<p>In this paper, we introduce an Adaptive Graph Signal Processing with Dynamic Semantic Alignment (AGSP-DSA) framework to perform robust multimodal data fusion across heterogeneous sources, including text, audio, and images. The proposed approach uses a dual-graph construction to learn both intra-model and inter-modal relations, spectral graph filtering to enhance informative signals, and effective node embeddings via Multi-scale Graph Convolutional Networks. In the semantic-aware attention mechanism, each modality may dynamically contribute to the context with respect to contextual relevance. The experimental outcomes on three benchmark datasets, including Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity dataset, Audio-Visual Event dataset, and MultiModal Internet Movie Database dataset, show that Adaptive Graph Signal Processing with Dynamic Semantic Alignment performs as the state of the art. More precisely, it achieves 95.3% accuracy, 93.6% F1 (Harmonic Mean of Precision and Recall) score, and 92.4% mean average precision on the Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity dataset, improving the MultiModal Graph Neural Network by 2.6% in accuracy. It gets 93.4% accuracy and 91.1% F1 score on Audio-Visual Event dataset, and 91.8% accuracy and 88.6% F1 score on MultiModal Internet Movie Database dataset, which demonstrates good generalization and robustness in the missing modality setting. These findings verify the efficiency of the proposed AGSP-DSA in promoting multimodal learning in sentiment analysis, event recognition, and multimedia classification.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Adaptive graph signal processing for robust multimodal fusion with dynamic semantic alignment

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月20日

查看全文
  • K. V. Karthikeya,
  • Arun Sekar Rajasekaran,
  • Ashok Kumar Das,
  • Vivekananda Bhat K,
  • Shantanu Pal

摘要

In this paper, we introduce an Adaptive Graph Signal Processing with Dynamic Semantic Alignment (AGSP-DSA) framework to perform robust multimodal data fusion across heterogeneous sources, including text, audio, and images. The proposed approach uses a dual-graph construction to learn both intra-model and inter-modal relations, spectral graph filtering to enhance informative signals, and effective node embeddings via Multi-scale Graph Convolutional Networks. In the semantic-aware attention mechanism, each modality may dynamically contribute to the context with respect to contextual relevance. The experimental outcomes on three benchmark datasets, including Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity dataset, Audio-Visual Event dataset, and MultiModal Internet Movie Database dataset, show that Adaptive Graph Signal Processing with Dynamic Semantic Alignment performs as the state of the art. More precisely, it achieves 95.3% accuracy, 93.6% F1 (Harmonic Mean of Precision and Recall) score, and 92.4% mean average precision on the Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity dataset, improving the MultiModal Graph Neural Network by 2.6% in accuracy. It gets 93.4% accuracy and 91.1% F1 score on Audio-Visual Event dataset, and 91.8% accuracy and 88.6% F1 score on MultiModal Internet Movie Database dataset, which demonstrates good generalization and robustness in the missing modality setting. These findings verify the efficiency of the proposed AGSP-DSA in promoting multimodal learning in sentiment analysis, event recognition, and multimedia classification.