4D millimeter-wave radar, an emerging sensor capturing spatial and Doppler velocity information, has gained significant attention in autonomous driving. However, its performance in 3D object detection remains limited due to its sparsity of point clouds and susceptibility to noise. Recent studies have explored temporal multi-frame fusion of radar point clouds to mitigate sparsity, but it often leads to depth distortions that hinder accurate object localization in multi-modal fusion approaches. To address these limitations, we propose a novel radar-camera fusion framework, MDGFusion, which contains two key innovations: (1) an Adaptive Radar Virtual Points Generation Module (ARGM) that generates depth-reliable dense virtual radar points by filtering anomaly depth information through segmentation-derived ROIs and coarse semantic labels; and (2) a Dual-Stream BEV Enhancing Fusion Module (DBEM) that employs multi-attention to fuse complementary radar and camera features, combining radar’s precise depth cues and camera’s rich semantics through dual-stream feature enhancement and dynamic modality weighting. Our framework significantly improves 3D object detection accuracy and enhances robustness under adverse illumination. Extensive experiments on VoD and TJ4DRadSet datasets demonstrate that our method achieves state-of-the-art results: 60.20% and 80.04% mAP in VoD’s EAA and driving corridor regions, and 38.46%/44.04% of 3D/BEV mAP on TJ4DRadSet.