Fake news detection is a major challenge in cyberspace governance. In this study, to address some of the challenges in multimodal fake news detection, such as the inadequacy of image feature modeling, the neglect of frequency domain information, and the problem of deep interaction and fusion of multimodal features, we propose a multimodal fake news detection model called image Frequency and Spatial domain analysis with Deep dynamic Trade-off fusion network Model (FSDTM). Firstly, the FSDM uses the discrete cosine transform (DCT) to transform the image to the frequency domain. Subsequently, the frequency and spatial domains of the image are modeled hierarchically using a convolutional neural network (CNN) and Transformer networks, and a gated information fusion network is designed to gradually fuse information from the frequency and spatial domains. Secondly, we constructed a diversified perceptual modal interaction layer and a deep dynamic trade-off fusion network. This facilitates effective interaction among different modalities and deep fusion of features. Additionally, the dynamic trade-off fusion network effectively eliminates redundant information, ensuring robust and efficient multimodal analysis. FSDTM is validated on three public datasets, Weibo, Twitter, and Pheme, and achieved the best performance of 90.9%, 95.1%, and 91.3% accuracy, respectively.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Multimodal Fake News Detection Model Leveraging Image Frequency and Spatial Domain Analysis with Deep Dynamic Trade-Off Fusion

  • Zhuzhu Zhang,
  • Xian Fu,
  • Tianrui Wu,
  • Yu Sun,
  • Ningning Zhang,
  • Hui Zhang

摘要

Fake news detection is a major challenge in cyberspace governance. In this study, to address some of the challenges in multimodal fake news detection, such as the inadequacy of image feature modeling, the neglect of frequency domain information, and the problem of deep interaction and fusion of multimodal features, we propose a multimodal fake news detection model called image Frequency and Spatial domain analysis with Deep dynamic Trade-off fusion network Model (FSDTM). Firstly, the FSDM uses the discrete cosine transform (DCT) to transform the image to the frequency domain. Subsequently, the frequency and spatial domains of the image are modeled hierarchically using a convolutional neural network (CNN) and Transformer networks, and a gated information fusion network is designed to gradually fuse information from the frequency and spatial domains. Secondly, we constructed a diversified perceptual modal interaction layer and a deep dynamic trade-off fusion network. This facilitates effective interaction among different modalities and deep fusion of features. Additionally, the dynamic trade-off fusion network effectively eliminates redundant information, ensuring robust and efficient multimodal analysis. FSDTM is validated on three public datasets, Weibo, Twitter, and Pheme, and achieved the best performance of 90.9%, 95.1%, and 91.3% accuracy, respectively.