Scholarlink

<p>The increasing sophistication of multimodal disinformation driven by generative AI and cross-modal manipulation poses significant challenges to the reliability of existing Multimodal fake news detection (MFND) systems. This work presents a rigorous interrogation and comprehensive evaluation of canonical MFND architectures, rather than proposing a new system. We assess the feasibility and robustness of five diverse architectures, LongBERT-VGG, CNN-VGG, LSTM-VGG, BERT-CLIP, and the end-to-end transformer ViLBERT across three benchmark datasets: D1, RECOVERY, and PolitiFact, which collectively represent varying degrees of class balance and content characteristics. Our analysis transcends conventional evaluations by subjecting these models to an extensive battery of adversarial perturbations: ten distinct image-based attacks and seven sophisticated text-based attacks. Crucially, we introduce novel adversarial textual manipulations generated via instruction-tuned LLMs, specifically comparing the potency of attacks derived from Meta’s Llama 3 8B against those from DeepSeek-LLM 7B Chat. We further introduce an analysis of black-box transfer attacks to simulate real-world threats. Furthermore, we systematically investigate the performance implications of modality ablation, providing granular insights into the functional interdependence of textual and visual features. Our empirical findings, now reported as mean ± standard deviation over multiple seeds and validated with statistical significance tests, illuminate significant vulnerabilities inherent in contemporary MFND models, particularly when confronted with advanced LLM-driven textual attacks. Notably, our results indicate that adversarial text generated by the DeepSeek model exhibits a statistically significant superior capacity to degrade detection performance compared to Llama 3 8B. The analysis further underscores the pivotal, albeit context-dependent, role of the visual modality. We supplement our quantitative findings with a qualitative error analysis using Grad-CAM, revealing model failure modes, including over-reliance on dataset artifacts. We also provide a practical analysis of computational costs and potential defense strategies. By advancing the frontiers of adversarial resilience and modality dependence analysis in MFND, this work establishes a foundational benchmark for the conception and validation of next-generation disinformation detection systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A rigorous interrogation of multimodal fake news detection paradigms: adversarial robustness and modality dependence

  • 期刊论文

  • 发表日期：

    2026年6月3日

查看全文
  • Aman Jolly,
  • Shailender Kumar

摘要

The increasing sophistication of multimodal disinformation driven by generative AI and cross-modal manipulation poses significant challenges to the reliability of existing Multimodal fake news detection (MFND) systems. This work presents a rigorous interrogation and comprehensive evaluation of canonical MFND architectures, rather than proposing a new system. We assess the feasibility and robustness of five diverse architectures, LongBERT-VGG, CNN-VGG, LSTM-VGG, BERT-CLIP, and the end-to-end transformer ViLBERT across three benchmark datasets: D1, RECOVERY, and PolitiFact, which collectively represent varying degrees of class balance and content characteristics. Our analysis transcends conventional evaluations by subjecting these models to an extensive battery of adversarial perturbations: ten distinct image-based attacks and seven sophisticated text-based attacks. Crucially, we introduce novel adversarial textual manipulations generated via instruction-tuned LLMs, specifically comparing the potency of attacks derived from Meta’s Llama 3 8B against those from DeepSeek-LLM 7B Chat. We further introduce an analysis of black-box transfer attacks to simulate real-world threats. Furthermore, we systematically investigate the performance implications of modality ablation, providing granular insights into the functional interdependence of textual and visual features. Our empirical findings, now reported as mean ± standard deviation over multiple seeds and validated with statistical significance tests, illuminate significant vulnerabilities inherent in contemporary MFND models, particularly when confronted with advanced LLM-driven textual attacks. Notably, our results indicate that adversarial text generated by the DeepSeek model exhibits a statistically significant superior capacity to degrade detection performance compared to Llama 3 8B. The analysis further underscores the pivotal, albeit context-dependent, role of the visual modality. We supplement our quantitative findings with a qualitative error analysis using Grad-CAM, revealing model failure modes, including over-reliance on dataset artifacts. We also provide a practical analysis of computational costs and potential defense strategies. By advancing the frontiers of adversarial resilience and modality dependence analysis in MFND, this work establishes a foundational benchmark for the conception and validation of next-generation disinformation detection systems.