To address the problems of noise interference, small target size, and complicated backgrounds in ship detection using synthetic aperture radar (SAR), this study proposes the GMS-YOLO, a lightweight model. This study uses the lightweight GhostConv to reduce computation and model parameters. To enhance feature representation while maintaining the model’s lightweight design, this study designs a C2f-M module based on the mixed local channel attention (MLCA) mechanism. Furthermore, SPPFAS is intended to improve detection performance for targets at various scales. Lastly, this study introduces the Wise-IoU to resolve the issue of unfavorable gradients caused by occurrences of low quality in the training dataset for target detection. The findings of the experiment show that, compared to YOLOv8n, the proposed GMS-YOLO algorithm achieved higher R, mAP50, and mAP50-95 on the SSDD dataset by 2.4%, 0.7%, and 1.3%, and on the HRSID dataset by 1.4%, 1%, and 1.1%. Additionally, the parameters and FLOPs were reduced to 2.8M and 7.8G, respectively, for both datasets. In the comparative experiments with other deep learning approaches, the GMS-YOLO algorithm still performed better, achieving a good balance between lightweight and accuracy.