With the rapid development of autonomous driving technology, the demand for real-time and accurate traffic sign detection has been increasing. To address the issue of missed detections of small objects and occluded signs in complex scenarios, this paper proposes an improved YOLOv5 model based on the fusion of CBAM (Convolutional Block Attention Module) and BiFPN (Bidirectional Feature Pyramid Network). By introducing CBAM to enhance the model’s ability to focus on key features and combining it with BiFPN to optimize the efficiency of multi-scale feature fusion, the detection robustness in complex environments is significantly improved. Experiments on the CTSD dataset show that the improved model achieves an average precision (mAP) of 92.5%, which is a 2.1% improvement over the original YOLOv5, while maintaining real-time detection speed (119 FPS). This research provides an efficient solution for the engineering application of traffic sign detection.