Accurate real-time monitoring of wheat head growth is crucial for effective agricultural management. However, achieving this is challenging due to the dense distribution of wheat heads in images, which often results in significant overlap, adhesion and occlusion. Furthermore, complex backgrounds and varying illumination conditions further exacerbate these challenges, making it difficult to achieve high accuracy in models used on portable devices. This paper introduces a lightweight version of the You-Only-Look-Once version 5 (YOLOv5) model that achieves high accuracy in wheat head detection while significantly reducing computational complexity and model parameters, making it well-suited for portable devices. To achieve these optimizations, we replaced specific convolutional, batch normalization, and SiLU (CBS) blocks, and bottlenecks in the C3 modules with Spatial-Channel Decoupled Downsampling (SCDown) and Compact Inverted Block (CIB). Furthermore, we integrated a Global Contextual (GC) block with the C3 module in the backbone network to enhance model accuracy. The use of CIB helps to deal with redundant information, and its variant, denoted CIB-H, increases the receptive field, further improving accuracy while reducing the model’s parameters and computational load. As a result, our proposed model achieves approximately \(90.6\%\) mAP@0.5 on the GWHD2021 dataset with only 1.2 M parameters and 3.7 GFLOPs.