<p>To achieve efficient and accurate detection of casting surface defects, this paper proposes FSD-DETR, a casting surface defect detection model based on real-time detection transformer (RT-DETR). First, the model adopts StarNet as the backbone network, replacing the original ResNet18, which effectively reduces both computational cost and parameter count, while significantly improving object detection accuracy. Second, frequency channel attention (FCA) is introduced during the feature extraction stage of the StarNet backbone to enhance the perception of small-scale defect targets by optimizing feature interaction. Additionally, a Feature Reuse Adaptive (FRA) module is designed and integrated into the CCFF component to strengthen multi-scale feature fusion, thereby improving detection performance. Finally, the GIoU loss is replaced with PIoU, which effectively suppresses false overlaps between background and target boxes, further enhancing detection accuracy and accelerating model convergence. In terms of performance evaluation, FSD-DETR achieves a mAP@50 of 96.1% on the CSDD dataset, representing a 1.7% improvement over the original RT-DETR model and a 2.1% to 6.8% gain compared to other classical detection models. Meanwhile, the model’s parameters and GFLOPs are reduced by 34.2% and 34.9%, respectively, fully meeting the stringent requirements for real-time and high-precision casting surface defect detection. Experimental results on the publicly available NEU-DET dataset further confirm the model’s strong generalization ability, with a 5.3% improvement in detection performance.</p>