Joint Decision Network with Modality-Specific and Dual Interactive Features for Fake News Detection
摘要
Fake news detection has achieved considerable progress in recent years, especially since multi-modal information was considered. However, most methods concentrate on feature-level inter-modal interaction and fusion, without considering the gap between feature-level operation and the high-level detection task. Furthermore, how to explore useful information and fully utilize both the modality-specific and cross-modal interactive features has not been well studied. In this paper, we present a novel approach named Joint Decision Network (JDN) with modality-specific and dual interactive features for multi-modal fake news detection, which includes an information-completeness-aware modality-specific feature extraction (IMFE) module, a local and global cross-modal interaction (LGCI) module, and a decision-level adaptive weighted fusion (DAWF) module. The IMFE module is designed to extract modality-specific features and preserve useful information while removing irrelevant information for classification. To explore the commonness among modalities and deal with the modality gap issue, the LGCI module performs dual inter-modal interaction. Specifically, local interaction is conducted by product between image and text representations, and global interaction is conducted by cross-modal graph propagation. The DAWF module performs weighted fusion on the predicted probabilities of the modality-specific and interactive features with weights by optimizing weighted classification loss to fully utilize the modality-specific and interactive features. Experiments on Weibo, Fakeddit, and Twitter datasets show that JDN can significantly outperform state-of-the-art related works.