Scholarlink

<p>To effectively integrate the structural relationships between data into the generated cross-modal features, numerous cross-modal retrieval methods have employed Graph Convolutional Networks (GCNs) for cross-modal feature learning. However, most studies independently learn features for different modalities, ultimately limiting structural consistency across cross-modal features. As text features encompass semantic structural information, we introduce a Text Semantic Structure-Guided Correlation Learning (TSSCL) method. This method leverages text features as supervision information to guide the learning of both image and label correlations. As a result, we can align the structural relationships within the common representations and label embeddings with those inherent in the texts. Furthermore, we introduce a novel Structure Transfer Graph Convolutional Network (STGCN) to maintain the global structural relationships across images. Additionally, we propose a series of semantic consistency losses and structural InfoNCE losses, which are beneficial for maintaining both the semantic consistency and structural consistency of the common representations and label embeddings. We perform experiments on the NUS-WIDE, MIRFlickr-25K, and MS-COCO datasets, and the results demonstrate that our TSSCL outperforms the current state-of-the-art cross-modal retrieval methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Text semantic structure-guided correlation learning for cross-modal retrieval

  • 期刊论文

  • 发表日期：

    2025年5月6日

查看全文
  • Jie Zhu,
  • Jingjing Fan,
  • Jianguang Zhao,
  • Shufang Wu,
  • Jianan Liu

摘要

To effectively integrate the structural relationships between data into the generated cross-modal features, numerous cross-modal retrieval methods have employed Graph Convolutional Networks (GCNs) for cross-modal feature learning. However, most studies independently learn features for different modalities, ultimately limiting structural consistency across cross-modal features. As text features encompass semantic structural information, we introduce a Text Semantic Structure-Guided Correlation Learning (TSSCL) method. This method leverages text features as supervision information to guide the learning of both image and label correlations. As a result, we can align the structural relationships within the common representations and label embeddings with those inherent in the texts. Furthermore, we introduce a novel Structure Transfer Graph Convolutional Network (STGCN) to maintain the global structural relationships across images. Additionally, we propose a series of semantic consistency losses and structural InfoNCE losses, which are beneficial for maintaining both the semantic consistency and structural consistency of the common representations and label embeddings. We perform experiments on the NUS-WIDE, MIRFlickr-25K, and MS-COCO datasets, and the results demonstrate that our TSSCL outperforms the current state-of-the-art cross-modal retrieval methods.