Research on feature fusion-based deep adversarial hashing for cross-modal retrieval
摘要
Current cross-modal retrieval methods predominantly depend on feature extraction techniques such as convolutional neural networks (CNNs), vision transformers (ViT), and contrastive language-image pre-training (CLIP). While ViT excels at capturing long-range dependencies, it often overlooks fine-grained local features. Conversely, CNNs are effective in extracting local patterns but may fall short in constructing comprehensive global representations. Thus, we propose a hybrid network integrating ViT and channel focused network (CFN) for enhanced multimodal representation. Moreover, hashing methods are widely used in cross-modal retrieval due to the high efficiency and low storage overhead, yet suffer from cross-modal hash code heterogeneity and semantic loss during dimensionality reduction. To address these issues, we propose a feature fusion-based deep adversarial hashing (FFDAH) method, which incorporates an efficient multimodal feature fusion module that leverages a Transformer encoder for deep encoding of multimodal data to mine semantic information. Experiments on three benchmark datasets demonstrate that FFDAH outperforms state-of-the-art methods.