To address the challenges faced by existing Deepfake detection methods in effectively highlighting and distinguishing forgery details, a dual-branch network is proposed, comprising a high-frequency feature enhancement branch and a high-frequency feature suppression branch. By integrating high-frequency features into the inputs of both branches, the separation of frequency-domain features from spatial ones is avoided during training, thereby allowing a more comprehensive information set to be retained. The architecture is structured into three hierarchical stages—initial, intermediate, and advanced—to progressively enable feature extraction and refinement. Subtle forgery traces that might be overlooked by conventional methods are captured through differential features, which are obtained by subtracting one branch’s features from the other. Additionally, intermediate texture difference attention fusion modules and advanced texture difference attention fusion modules are introduced to strategically aggregate multi-stage features. The superior performance of the proposed model is demonstrated through extensive experiments, showing significant advancements in Deepfake detection tasks.