Synthetic Aperture Radar (SAR) imaging is a critical technology in remote sensing, providing high-resolution images under all weather conditions and lighting scenarios. However, SAR image interpretation and object detection remain challenging due to inherent noise and the lack of visual features. In this paper, we propose a novel framework, Multimodal Bind for Few-Shot SAR Object Detection (MMB-Det), which addresses these challenges by leveraging the pretrained CLIP model to align visual embeddings from optical (OPT), infrared (IR), and SAR modalities into a unified embedding space. The Multimodal Alignment module uses contrastive loss to ensure embeddings from different modalities of the same object are close, enhancing cross-modal semantic correlations. The Contrastive Multimodal Priors module employs OPT and IR images as support inputs to guide the SAR object detection task, effectively transferring knowledge from these richer modalities. We validate our approach on a newly curated dataset, MMB-FS, designed for few-shot SAR object detection, demonstrating superior performance over state-of-the-art methods. Our results highlight the effectiveness of multimodal alignment and contrastive learning in improving SAR detection capabilities in few-shot scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Bind for Few-Shot SAR Object Detection in Drone Remote Sensing Images

  • Xiyu Qi,
  • Kunyu Yang,
  • Hanru Shi,
  • Yichang Luo,
  • Lei Ge,
  • Yingbing Ma,
  • Yunping Ge

摘要

Synthetic Aperture Radar (SAR) imaging is a critical technology in remote sensing, providing high-resolution images under all weather conditions and lighting scenarios. However, SAR image interpretation and object detection remain challenging due to inherent noise and the lack of visual features. In this paper, we propose a novel framework, Multimodal Bind for Few-Shot SAR Object Detection (MMB-Det), which addresses these challenges by leveraging the pretrained CLIP model to align visual embeddings from optical (OPT), infrared (IR), and SAR modalities into a unified embedding space. The Multimodal Alignment module uses contrastive loss to ensure embeddings from different modalities of the same object are close, enhancing cross-modal semantic correlations. The Contrastive Multimodal Priors module employs OPT and IR images as support inputs to guide the SAR object detection task, effectively transferring knowledge from these richer modalities. We validate our approach on a newly curated dataset, MMB-FS, designed for few-shot SAR object detection, demonstrating superior performance over state-of-the-art methods. Our results highlight the effectiveness of multimodal alignment and contrastive learning in improving SAR detection capabilities in few-shot scenarios.