Adversarial patch attacks threaten the reliability of object detectors by causing severe misclassifications, especially in safety-critical environments. In this work, we propose a comprehensive defense pipeline that not only restores detection performance but also significantly improves it. Our method leverages a latent diffusion model to recover semantically coherent regions affected by adversarial patches, leading to confidence gains of +26.61% for YOLOv5 and +26.91% for YOLOv7 - exceeding the models’ original predictions on clean images. In contrast to prior approaches that merely attempt restoration, we demonstrate that diffusion models can enhance object detection performance under attack, while maintaining practical efficiency with of inference time. We also present an optimized attack strategy based on EigenCAM and grid search, which identifies and targets the most vulnerable regions of the image. Experimental results show that our method consistently outperforms classical and recent defenses such as JPEG compression, spatial smoothing, SAC [17], and DIFFender [8], both in robustness and in detection confidence recovery. These findings highlight the potential of generative models not only for defense but for strengthening object detectors in adversarial scenarios.