In order to solve the problems of many defect types, small targets, and easy false detection and missed detection in various complex backgrounds, an improved insulator defect detection algorithm of YOLO11s is proposed. In order to solve the problem of small targets and complex backgrounds in defect detection, the PConv convolution module is introduced, which has significant advantages in low-contrast object detection tasks. Secondly, an improved CBAM attention mechanism is introduced to enhance the feature extraction ability of small targets and the ability to suppress complex backgrounds. In order to deploy mobile inspection equipment with limited computing power such as drones and reduce computing overhead, this paper improves the SimAM module, and the SimAM-E module and the improved CBAM module form a network structure with dual attention mechanism, which greatly improves the model performance. In addition, in order to solve the problem of low fault tolerance of some small target defect types, the original CIoU loss function is replaced with the En-Shape-IoU loss function. The experimental results show that the accuracy of the proposed method is improved by 2.86%, the recall rate is similar to that of the original model, the mAP 50 value is increased by 2.09%, and the mAP 50–95 value is increased by 2.566%, which effectively improves the target detection ability of small targets and extreme backgrounds.