Scholarlink

<p>To tackle the challenges of cross-modal fine-grained alignment and matching in text-based person re-identification, this paper proposes a novel model called interlaced cross-attention and LoRA fine-tuning (ICLF). The model is grounded in the encoder–decoder architecture, facilitating both coarse-grained and fine-grained alignment of image and text features. Specifically, for the encoder, we efficiently fine-tune the encoder using the LoRA mechanism, to minimize computational resource consumption. Coarse-grained alignment is achieved through image-text contrastive learning, narrowing the gap between single-modal outputs prior to deep interaction. For the decoder, we propose an interlaced cross-modal decoder structure designed to enhance bidirectional interaction between text and images by swapping queries and key-value pairs, achieving fine-grained alignment through image-text matching supervised by a joint modal soft-label which fully leverages the similarity distilled knowledge of intramodal and implicit alignment through masked language modeling. Our empirical evaluations are conducted on three datasets, including CUHK-PEDES, ICFG-PEDES, and RSTPReid, demonstrating the effectiveness of our method. Compared to previous methods, ICLF notably improves fine-grained alignment accuracy and computational efficiency, leading to enhanced performance in text-based person re-identification tasks.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Fine-grained text-based person re-identification via interlaced cross-attention and LoRA fine-tuning

  • 期刊论文

  • 发表日期：

    2025年5月27日

查看全文
  • Mengnan Hu,
  • Wenjing Zhang,
  • Qianli Zhou,
  • Rong Wang

摘要

To tackle the challenges of cross-modal fine-grained alignment and matching in text-based person re-identification, this paper proposes a novel model called interlaced cross-attention and LoRA fine-tuning (ICLF). The model is grounded in the encoder–decoder architecture, facilitating both coarse-grained and fine-grained alignment of image and text features. Specifically, for the encoder, we efficiently fine-tune the encoder using the LoRA mechanism, to minimize computational resource consumption. Coarse-grained alignment is achieved through image-text contrastive learning, narrowing the gap between single-modal outputs prior to deep interaction. For the decoder, we propose an interlaced cross-modal decoder structure designed to enhance bidirectional interaction between text and images by swapping queries and key-value pairs, achieving fine-grained alignment through image-text matching supervised by a joint modal soft-label which fully leverages the similarity distilled knowledge of intramodal and implicit alignment through masked language modeling. Our empirical evaluations are conducted on three datasets, including CUHK-PEDES, ICFG-PEDES, and RSTPReid, demonstrating the effectiveness of our method. Compared to previous methods, ICLF notably improves fine-grained alignment accuracy and computational efficiency, leading to enhanced performance in text-based person re-identification tasks.