While rice pests and diseases significantly impact crop yields, existing deep learning methods for their detection face challenges with accuracy and deployment complexity. Addressing these issues, this study proposes the YOLOv8-HSFPN, an advanced detection framework. Firstly, it features an innovative High-level Select Feature Pyramid Network (HSFPN) neck network that effectively integrates high-level and low-level feature sets for enhanced feature fusion. Secondly, the addition of a deformable self-attention module further refines the model’s adaptability to the varying shapes and locations of targets, dynamically adjusting to the salient features. The proposed model has undergone comparative and ablation studies alongside YOLOv8, YOLOv9, and YOLOv5, confirming its improved accuracy and streamlined deployment. This integration results in a robust detection model that not only marks a significant leap in accuracy, evidenced by a 3% empirical increase over the standard YOLOv8, but is also remarkably compact. At a mere 3.97MB, this substantial 49.87% size reduction compared to its predecessors renders it exceptionally suitable for devices with limited computational resources, thereby enhancing its viability in practical, real-world applications.