Multimodal Bind for Few-Shot SAR Object Detection in Drone Remote Sensing Images
摘要
Synthetic Aperture Radar (SAR) imaging is a critical technology in remote sensing, providing high-resolution images under all weather conditions and lighting scenarios. However, SAR image interpretation and object detection remain challenging due to inherent noise and the lack of visual features. In this paper, we propose a novel framework, Multimodal Bind for Few-Shot SAR Object Detection (MMB-Det), which addresses these challenges by leveraging the pretrained CLIP model to align visual embeddings from optical (OPT), infrared (IR), and SAR modalities into a unified embedding space. The Multimodal Alignment module uses contrastive loss to ensure embeddings from different modalities of the same object are close, enhancing cross-modal semantic correlations. The Contrastive Multimodal Priors module employs OPT and IR images as support inputs to guide the SAR object detection task, effectively transferring knowledge from these richer modalities. We validate our approach on a newly curated dataset, MMB-FS, designed for few-shot SAR object detection, demonstrating superior performance over state-of-the-art methods. Our results highlight the effectiveness of multimodal alignment and contrastive learning in improving SAR detection capabilities in few-shot scenarios.