Recently, object detection based Unmanned Aerial Vehicles (UAVs) plays a wide role in many real-life scenarios, such as agriculture and transportation. Although UAVs have been widely used, the detection of small targets is ineffective in conventional object detection networks due to extensive scenes and abundant information. To solve the problems of small object detection, we proposed a novel Bi-directional Extends Feature Pyramid Network (Bi-EFPN), which deepens the depth of Feature Pyramid Network (FPN) and performs better in feature fusion. The Gated Channel and Position Attention Model (GCPA) is introduced to capture more details of small objects in large-sized feature maps. Considering that the Bi-EFPN and GCPA will cost more resources in the neck, we proposed a lightweight detection head with Partial Attention (PAHead), which reduced the computational complexity by approximately 6.8 GFLOPs while slightly improving detection accuracy. Finally, it shows great performance on the visdrone2019 dataset, the mean average precision is improved by 5.1% and the parameters decreased by approximately 10% in contrast to YOLOv8s. Meanwhile, good generalization performance was demonstrated on the Tinyperson dataset and HIT-UAV dataset. The comprehensive experiments indicate that our network has great potential in small object detection.