Deep explainable multimodal framework for detecting war-related misinformation in conflict zones in crisis communication
摘要
Misinformation in conflict zones poses significant risks to public trust, humanitarian response, and crisis communication. Existing approaches to misinformation detection often rely on either textual or visual cues in isolation, limiting their effectiveness in capturing the multimodal nature of misinformation circulated through modern social media. Moreover, these models frequently operate as black boxes, providing limited interpretability and failing to address the urgent need for transparency in high-stakes domains. To address these limitations, this study proposes an explainable multimodal framework that integrates text and image modalities for misinformation detection in war-related contexts. The framework leverages AraBERT for textual feature extraction and vision transformer for visual representation, with a cross-modal transformer employed to fuse heterogeneous features. To enhance interpretability, state-of-the-art explainability methods such as LIME and Grad-CAM are incorporated, allowing stakeholders to understand the contribution of each modality in the decision-making process. Extensive experiments were conducted on multiple datasets, including FakeNewsNet, Fakeddit, and two custom collections from Twitter and Instagram. The proposed model achieved superior performance, with accuracy rates of 97.89%, 98.34%, 98.89%, and 97.63% on these datasets, respectively. Beyond predictive performance, the model also demonstrated high explainability through evaluation on explainability fidelity score, Trustworthiness Index, and counterfactual faithfulness score. These results confirm that the proposed approach not only surpasses existing baselines but also builds trust by ensuring transparency in automated misinformation detection.