The MobileOne-YOLO lightweight detection model is put forward for surface defect detection of wafer. Based on the YOLOv7 model, the backbone network is replaced with the MobileOne network. The re-parameterization and refocus convolution (RefConv) module and the coordinate attention (CA) mechanism are introduced into the feature pyramid network (FPN) to enhance feature extraction and strengthen inter-channel connections. As a result, the model performance is further improved. Ablation experiments on a self-collected material dataset validate that the improved algorithm surpasses current algorithms in both detection accuracy and speed.