To deal with the problems of missed detection and false detection in pedestrian detection in low-light and occluded scenes, this paper proposes a dual-modal fusion detection scheme. This method is based on edge attention guidance and multi-scale fusion to achieve deep fusion of infrared images and visible images. It uses yolov5 as the model base and integrates adaptive rotation convolution to promote the feature extraction efficiency. Besides, a channel attention module is introduced in the backbone network, and replace the original loss function with Inner-IoU. Experiments on the LLVIP dataset indicate that the detection precision of this modus is 96.2%, the recall is 89%, the mAP50 is 95.6%, and the mAP50–95 is 64.7%. Compared with the original single-modal infrared image model, mAP50 and mAP50–95 increased by 2.1% and 5.5%, showing excellent detection accuracy.