Universal forged image detection and localization via self-supervised data generation and large-scale model adaptation
摘要
Image forgery poses a serious threat to public security. Real-world forged images, often created through diverse tools and multi-stage post-processing, exhibit complex statistical patterns, making it extremely challenging for researchers to model forged behavior comprehensively. Current dominant image forensic models are predominantly based on deep learning, yet such approaches face a dilemma: small-scale training datasets lead to poor real-world generalization and robustness, while large-scale training samples are unattainable due to the labor-intensive nature of image manipulation. To address this issue, this paper tackles two core challenges: 1. We introduce a self-supervised training strategy to efficiently generate large-scale, high-quality forged samples; 2. We leverage a pre-trained large model, Segment Anything Model (SAM), to enhance generalization and robustness. Our self-supervised strategy employs three steps: SAM-based object extraction, automated forgery generation via editing tools, and mask synthesis. Our forensic model integrates SAM’s generalized object features and steganalysis-based high-dimensional features, leveraging SAM’s segmentation capabilities for effective forensics. Additionally, existing image forensic methods typically assume that the input images are already forged, focusing solely on forgery localization, which does not reflect real-world scenarios. Our model can simultaneously perform image forgery detection and localization. Experiments show that our proposed model achieves state-of-the-art performance and robustness across multiple natural scene benchmarks, even when trained on small-scale datasets. It’s worth noting that our model also achieved state-of-the-art performance even on real-world document image forgery datasets, which significantly differ from natural images, showcasing the exceptional generalization capability of our approach.