Traffic surveillance systems face challenges in vehicle detection in dense environments due to occlusions, pedestrians, and adverse conditions such as nighttime glare. Non-vehicle objects, including road signs and billboards, create noise and false positives, reducing detection accuracy and reliability. To address these issues, we propose a two-stage refinement framework: a pre-trained Co-DETR model eliminates irrelevant objects, followed by fine-tuned deep-learning models for precise vehicle detection. Additionally, detection stability is enhanced with Weighted Boxes Fusion (WBF), and image quality is improved through NAFNet for restoration and GSAD for low-light enhancement. Our approach significantly improves accuracy and robustness, achieving a mean Average Precision (mAP) of 0.9022 and a final score of 0.7779, which combines the F1 score and mAP, on the SoICT Hackathon 2024—Traffic Vehicle Detection Dataset.