Scholarlink

<p>With the emergence of large pre-trained vision-language models (VLMs), prompt learning methods like CoOp have recently demonstrated powerful capabilities in visual recognition and transfer learning across various downstream tasks. These methods introduce a set of learnable vectors as text prompts from the language modality to adapt VLMs to a wide range of downstream tasks. However, this unimodal design disrupts the original alignment paradigm between text and visual representations in the pre-trained models, resulting in suboptimal solutions. In this paper, we propose an attentive dual-modality prompt learning (ADPL), which adapts large pre-trained vision-language models to downstream tasks through interactive attention between text and visual prompts. To take into account all the learnable prompt layers, we further propose a dynamic coefficient generation module in ADPL to comprehensively consider the prompts acquired from previous layers. We evaluate the effectiveness of our method on three representative tasks: generalization to novel classes, new target datasets, and unseen domain shifts. Extensive experimental results on 11 datasets demonstrate the effectiveness and strong generalization capability of our approach. The code can be found at <a href="https://github.com/PromptLearningCoding/ADPL">https://github.com/PromptLearningCoding/ADPL</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Adpl: attentive dual-modality prompt learning for vision-language understanding

  • 期刊论文

  • 发表日期：

    2025年6月1日

查看全文
  • Zhiyong Deng,
  • Ronggui Wang,
  • Lixia Xue,
  • Juan Yang

摘要

With the emergence of large pre-trained vision-language models (VLMs), prompt learning methods like CoOp have recently demonstrated powerful capabilities in visual recognition and transfer learning across various downstream tasks. These methods introduce a set of learnable vectors as text prompts from the language modality to adapt VLMs to a wide range of downstream tasks. However, this unimodal design disrupts the original alignment paradigm between text and visual representations in the pre-trained models, resulting in suboptimal solutions. In this paper, we propose an attentive dual-modality prompt learning (ADPL), which adapts large pre-trained vision-language models to downstream tasks through interactive attention between text and visual prompts. To take into account all the learnable prompt layers, we further propose a dynamic coefficient generation module in ADPL to comprehensively consider the prompts acquired from previous layers. We evaluate the effectiveness of our method on three representative tasks: generalization to novel classes, new target datasets, and unseen domain shifts. Extensive experimental results on 11 datasets demonstrate the effectiveness and strong generalization capability of our approach. The code can be found at https://github.com/PromptLearningCoding/ADPL.