Retinal fundus images constitute a crucial foundation for the clinical diagnosis of ophthalmic diseases. However, image quality degradation often results in the omission of subtle pathological features, thereby undermining diagnostic accuracy and reliability. Many existing enhancement methods fail to effectively restore fine structural details. To address these limitations, this paper proposes a multi-scale detail restoration framework, MSDR-GAN, based on generative adversarial networks. The model introduces a spatial-frequency dual-domain feature extraction strategy, which leverages fast Fourier transform for global frequency-domain modeling, integrates dilated convolution for enhanced multi-scale spatial feature representation, and incorporates a channel attention mechanism to enable dynamic cross-domain feature fusion. Additionally, the model employs cascaded average pooling and residual learning to adaptively refine high-frequency texture details—such as vessel branches and the boundaries of the optic cup and disc, thereby mitigating such problems as artifact retention and image blurring. Extensive experiments, including comparisons with state-of-the-art approaches and ablation studies, validate the superiority of MSDR-GAN. Furthermore, downstream segmentation tasks on retinal vessels and optic structures confirm the method’s effectiveness in structural detail enhancement and its potential clinical applicability.