Driven by the swift advancements in drone technology, object detection methodologies have proliferated within the realm of intelligent drone transportation systems. The training data of existing unmanned aerial vehicle detection models mainly relies on visible light images, which leads to insufficient generalization ability caused by the singularity of data modality, resulting in a significant decrease in detection accuracy. In response to this issue, this article proposes a dual-mode aerial vehicle detection method VI-YOLO based on YOLOv8n, which combines the advantages of visible light image and infrared image data fusion. Firstly, based on YOLOv8n, a dual branch feature extraction network was designed using mid-term fusion; Then design a Bidirectional Feature Pyramid Network (BiFPN) feature fusion module to be applied in the neck network, so that the network can fully utilize the advantages of visible light and infrared images; Subsequently, the Efficient Multiscale Attention (EMA) mechanism was incorporated into the neck network, thereby augmenting its capacity for feature extraction. Thereafter, the Weighted Intersection over Union (WIoU) loss function was employed, which not only expedited the convergence rate of the bounding box regression loss but also enhanced the precision of the regression process. The experiment showed that the improved network on the DroneVehicle data machine achieved an average detection accuracy of 87.8%. Compared with the original model, the improved network is more effective in performing unmanned aerial vehicle detection tasks.