Diffusion models have recently emerged as powerful generative frameworks, yet their forgeries are often challenging to detect due to the absence of general artifacts typically seen in GAN-based images. In this paper, we rethink the denoising process of diffusion models and propose a novel forgery detection framework based on the intermediate features of a pretrained denoising neural network. Our approach captures the pixel-level variation trends between perturbed inputs and hidden representations across multiple timesteps, modeling the temporal evolution of internal activations. To enhance generalization, we propose a Structure-Aware Information Bottleneck that enforces spatially selective compression at the representation level, enabling the model to retain discriminative cues while effectively suppressing redundant signals. Experiments on multiple benchmarks demonstrate that our method is able to generalize across various diffusion-based forgeries, highlighting its effectiveness in detecting synthesized images under diverse generative conditions. Our code is available at https://github.com/ycfang-lab/RDP4DFD .