Rapid and accurate disaster classification is critical for effective emergency response and resource allocation. Existing approaches to multimodal crisis classification typically presume that the textual and visual elements of a tweet share the same label, thereby overlooking frequent mismatches between modalities and neglecting the contextual links that span multiple tweets. To address these limitations, we introduce GNN-MCC, a novel Graph Neural Network based framework for Multimodal Crisis Classification. Our method constructs a unified graph representation in which individual tweets are decomposed into separate text and image nodes, allowing the model to capture complex intra- and inter-modal relationships. To further reconcile discrepancies between modalities, we incorporate a contrastive attention mechanism that aligns complementary information across text and image channels. Empirical evaluation on the publicly available CrisisMMD dataset demonstrates that GNN-MCC substantially outperforms state-of-the-art baselines under both label-consistent and label-inconsistent conditions. The source code for this paper is available at https://github.com/kliean/GNN-MCC .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Crisis Classification via Graph Neural Networks

  • Kailing Shen,
  • Hongbin Wang,
  • Yantuan Xian,
  • Junjun Guo,
  • Zhengtao Yu

摘要

Rapid and accurate disaster classification is critical for effective emergency response and resource allocation. Existing approaches to multimodal crisis classification typically presume that the textual and visual elements of a tweet share the same label, thereby overlooking frequent mismatches between modalities and neglecting the contextual links that span multiple tweets. To address these limitations, we introduce GNN-MCC, a novel Graph Neural Network based framework for Multimodal Crisis Classification. Our method constructs a unified graph representation in which individual tweets are decomposed into separate text and image nodes, allowing the model to capture complex intra- and inter-modal relationships. To further reconcile discrepancies between modalities, we incorporate a contrastive attention mechanism that aligns complementary information across text and image channels. Empirical evaluation on the publicly available CrisisMMD dataset demonstrates that GNN-MCC substantially outperforms state-of-the-art baselines under both label-consistent and label-inconsistent conditions. The source code for this paper is available at https://github.com/kliean/GNN-MCC .