Scholarlink

<p>Text-video retrieval evaluates the semantic similarity between text and video within a shared embedding space. The performance of retrieval is often limited by the expressive power of representations. In addition, the model’s ability to understand the semantic associations between text and video is crucial for performance. To address the limitations posed by weak representation expressivity and the need for more nuanced semantic alignment, this paper proposes a text-video retrieval method termed <b>SERF</b> based on <b>S</b>emantic <b>E</b>nrichment and <b>R</b>econstruction <b>F</b>eedback. Specifically, we introduce two core modules designed to strengthen the feature representation and the semantic association between text and video, respectively. (1) We leverage a Multimodal Large Language Model (MLLM) to generate detailed video descriptions, thereby strengthening and enriching video-side features with fine-grained, event-level semantics. (2) We introduce semantic reconstruction feedback to serve as an auxiliary supervision alongside the primary global-local text-video alignment task, which provides feedback to strengthen the model’s robustness in capturing text-video interactions and is discarded during inference to maintain efficiency. Experimental results on MSRVTT, MSVD, and ActivityNet demonstrate the competitive performance of SERF, achieving R@1 scores of 48.7%, 49.6%, and 43.5%, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Strengthen-then-retrieve: semantic enrichment and reconstruction feedback for text-video retrieval

  • 期刊论文

  • 发表日期：

    2026年7月16日

查看全文
  • Jiayao Qian,
  • Shijie Luo,
  • Lili Yan,
  • Yan Liu

摘要

Text-video retrieval evaluates the semantic similarity between text and video within a shared embedding space. The performance of retrieval is often limited by the expressive power of representations. In addition, the model’s ability to understand the semantic associations between text and video is crucial for performance. To address the limitations posed by weak representation expressivity and the need for more nuanced semantic alignment, this paper proposes a text-video retrieval method termed SERF based on Semantic Enrichment and Reconstruction Feedback. Specifically, we introduce two core modules designed to strengthen the feature representation and the semantic association between text and video, respectively. (1) We leverage a Multimodal Large Language Model (MLLM) to generate detailed video descriptions, thereby strengthening and enriching video-side features with fine-grained, event-level semantics. (2) We introduce semantic reconstruction feedback to serve as an auxiliary supervision alongside the primary global-local text-video alignment task, which provides feedback to strengthen the model’s robustness in capturing text-video interactions and is discarded during inference to maintain efficiency. Experimental results on MSRVTT, MSVD, and ActivityNet demonstrate the competitive performance of SERF, achieving R@1 scores of 48.7%, 49.6%, and 43.5%, respectively.