Remote sensing image-text retrieval (RSITR) aims to retrieve relevant information from large-scale remote sensing (RS) data collected by uncrewed aerial vehicles and satellites, which has gained significant attention recently. Existing methods based on local or global feature matching only extract features from deep layers while ignoring the low-level details contained in shallow layers. These methods face limitations in capturing multi-object and detailed features of RS images, limiting the retrieval accuracy for RSITR. In this work, We propose a multi-level global-local fusion framework with a progressive curriculum learning strategy (MGLFCL) for remote sensing image-text retrieval to address this issue. First, we designed a Multi-level Global-Local Information Fusion Module (MGLF) to enhance feature representation. Second, we introduce a curriculum learning-based progressive alignment strategy (CLA) to improve model performance. Finally, we utilize the rich semantic knowledge of Vision-Language Pretraining (VLP) models by integrating a frozen Contrastive Language-Image Pretraining (CLIP) encoder. Comprehensive experimental results on the RSICD and RSITMD datasets demonstrate the effectiveness and superiority of our proposed method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Fine-Grained Remote Sensing Image-Text Retrieval via Multi-level Global-Local Feature Fusion

  • Pengfei Yuan,
  • Shan Zhao

摘要

Remote sensing image-text retrieval (RSITR) aims to retrieve relevant information from large-scale remote sensing (RS) data collected by uncrewed aerial vehicles and satellites, which has gained significant attention recently. Existing methods based on local or global feature matching only extract features from deep layers while ignoring the low-level details contained in shallow layers. These methods face limitations in capturing multi-object and detailed features of RS images, limiting the retrieval accuracy for RSITR. In this work, We propose a multi-level global-local fusion framework with a progressive curriculum learning strategy (MGLFCL) for remote sensing image-text retrieval to address this issue. First, we designed a Multi-level Global-Local Information Fusion Module (MGLF) to enhance feature representation. Second, we introduce a curriculum learning-based progressive alignment strategy (CLA) to improve model performance. Finally, we utilize the rich semantic knowledge of Vision-Language Pretraining (VLP) models by integrating a frozen Contrastive Language-Image Pretraining (CLIP) encoder. Comprehensive experimental results on the RSICD and RSITMD datasets demonstrate the effectiveness and superiority of our proposed method.