Multi-modal object detection has emerged as a critical research direction in recent years, as complementary characteristics from different modalities can effectively enhance detection accuracy and model robustness. This study proposes an innovative approach by investigating feature representation mechanisms through dual perspectives of temporal and frequency domains. In the frequency domain, differentiated filtering mechanisms are implemented to separately extract high-frequency edge information from infrared images and textural detail features from visible light images. The disentangled multi-modal features are subsequently integrated through a hierarchical fusion framework. Comprehensive experiments on the public multi-modal infrared-visible DroneVehicle dataset demonstrate the superior performance of our algorithm in small target detection tasks. The proposed method achieves state-of-the-art performance (mAP = 84.6%) while maintaining computational efficiency without incurring substantial computational overhead.