A Multimodal Fake News Detection Model Leveraging Image Frequency and Spatial Domain Analysis with Deep Dynamic Trade-Off Fusion
摘要
Fake news detection is a major challenge in cyberspace governance. In this study, to address some of the challenges in multimodal fake news detection, such as the inadequacy of image feature modeling, the neglect of frequency domain information, and the problem of deep interaction and fusion of multimodal features, we propose a multimodal fake news detection model called image Frequency and Spatial domain analysis with Deep dynamic Trade-off fusion network Model (FSDTM). Firstly, the FSDM uses the discrete cosine transform (DCT) to transform the image to the frequency domain. Subsequently, the frequency and spatial domains of the image are modeled hierarchically using a convolutional neural network (CNN) and Transformer networks, and a gated information fusion network is designed to gradually fuse information from the frequency and spatial domains. Secondly, we constructed a diversified perceptual modal interaction layer and a deep dynamic trade-off fusion network. This facilitates effective interaction among different modalities and deep fusion of features. Additionally, the dynamic trade-off fusion network effectively eliminates redundant information, ensuring robust and efficient multimodal analysis. FSDTM is validated on three public datasets, Weibo, Twitter, and Pheme, and achieved the best performance of 90.9%, 95.1%, and 91.3% accuracy, respectively.