To address the issues of large model size, limited scene adaptability, and deployment challenges in existing road pothole detection models, a lightweight multi-scene road pothole detection model, YOLOv5-LMPD, was proposed. In the dataset, traditional image processing methods are used to generate pothole images under different brightness conditions, enriching the diversity of the dataset. Next, replace the backbone of the baseline model with MobileNetV3 for a lightweight network model. Additionally, the ShuffleAttention mechanism was integrated into the head of YOLOv5, allowing the network to focus more on the target objects, thereby improving detection performance. Finally, the FocalEIoU loss function was adopted in place of CIoU, concentrating on medium-quality anchor boxes to enhance overall detection accuracy. Experimental results demonstrate that the YOLOv5-LMPD achieved a 2.9% increase in precision and a 0.3% improvement in mAP@0.5 on a custom pothole dataset. The modified algorithm is more lightweight, with enhanced accuracy, providing a technical reference for pothole detection in multiple lighting conditions.