Cross-modal hashing focuses on learning compact binary codes to enable efficient retrieval of relevant information across different modalities. Recent advances in vision-language pretraining, especially the Contrastive Language-Image Pretraining (CLIP) model, have demonstrated remarkable capabilities in aligning visual and textual representations. However, directly integrating CLIP into hashing frameworks remains underexplored. In this paper, we propose CLIP-Driven Deep Hashing (CDDH), a novel approach that leverages CLIP’s powerful multimodal representations to generate effective hash codes. Specifically, we employ CLIP’s pre-trained embeddings as feature representations and introduce a dual-stream hash learning network that refines these features for cross-modal retrieval. Our method incorporates contrastive learning, quantization constraints, and similarity-preserving objectives to enhance retrieval accuracy. Extensive experiments on standard cross-modal benchmarks, including NUS-WIDE and MIRFlickr-25K, demonstrate the better position of our method over state-of-the-art hashing approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

CLIP-Driven Deep Hashing for Cross-Modal Retrieval

  • Zhichao Han,
  • Azreen Azman,
  • Mas Rina Mustaffa,
  • Fatimah Khalid

摘要

Cross-modal hashing focuses on learning compact binary codes to enable efficient retrieval of relevant information across different modalities. Recent advances in vision-language pretraining, especially the Contrastive Language-Image Pretraining (CLIP) model, have demonstrated remarkable capabilities in aligning visual and textual representations. However, directly integrating CLIP into hashing frameworks remains underexplored. In this paper, we propose CLIP-Driven Deep Hashing (CDDH), a novel approach that leverages CLIP’s powerful multimodal representations to generate effective hash codes. Specifically, we employ CLIP’s pre-trained embeddings as feature representations and introduce a dual-stream hash learning network that refines these features for cross-modal retrieval. Our method incorporates contrastive learning, quantization constraints, and similarity-preserving objectives to enhance retrieval accuracy. Extensive experiments on standard cross-modal benchmarks, including NUS-WIDE and MIRFlickr-25K, demonstrate the better position of our method over state-of-the-art hashing approaches.