Fabric defects are characterized by a wide variety of types, varied scales, and similarity to the background. To address these challenges in detection, our research introduces a fabric defect detection algorithm named Yolo-DPW, which is an adaptation of the YOLOv8 model. This algorithm is designed to strike an optimal balance between detection accuracy and speed. The improved model in this research utilizes the PConv-C2 module to replace the C2f module in the original Backbone, which greatly reduces the number of parameters. The CIoU loss function used in the original YOLOv8 model is improved to WIoU loss function with dynamic non-monotonic focusing mechanism. At the same time, we use the D-Att module to improve the detection rate of defects with large differences in aspect ratio in the model. The experimental results show that the improved model improves the mean average precision (mAP) by 4.78% over the original model for the fabric defect detection task, while the detection speed reaches 203.17 FPS, which meets the detection requirements in industrial scenarios.