Scholarlink

<p>Prompt tuning for pre-trained language models (PLMs) has shown its effectiveness and superiority in few-shot learning. However, its success heavily depends on prompt engineering which designs different prompts for specific tasks. Continuous prompt templates can avoid this compared to discrete prompt templates, but how to obtain an optimal continuous prompt template is a question worth studying. In order to overcome this challenge, we focus on eliciting knowledge from PLMs and propose a novel <b>K</b>nowledge <b>C</b>ontrast-Enhanced Continuous <b>P</b>rompt <b>T</b>uning method (KCPT) based on contrastive learning. This method aims to obtain more meaningful and distinguishable prompt embeddings by introducing a contrastive learning strategy, enabling the model to better capture subtle differences between different prompts. Extensive experiments on 10 datasets demonstrate that our KCPT improves accuracy by 1–3% and stability by 26–40% on average compared to SOTA prompt tuning methods in low-resource settings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Knowledge Contrast-Enhanced Continuous Prompt Tuning for few-shot learning

  • 期刊论文

  • 发表日期：

    2025年5月7日

查看全文
  • Fei Li,
  • Youzhi Huang,
  • Yanyan Wang,
  • Zhengyi Chen,
  • Yin Xu,
  • Xiangyang Li

摘要

Prompt tuning for pre-trained language models (PLMs) has shown its effectiveness and superiority in few-shot learning. However, its success heavily depends on prompt engineering which designs different prompts for specific tasks. Continuous prompt templates can avoid this compared to discrete prompt templates, but how to obtain an optimal continuous prompt template is a question worth studying. In order to overcome this challenge, we focus on eliciting knowledge from PLMs and propose a novel Knowledge Contrast-Enhanced Continuous Prompt Tuning method (KCPT) based on contrastive learning. This method aims to obtain more meaningful and distinguishable prompt embeddings by introducing a contrastive learning strategy, enabling the model to better capture subtle differences between different prompts. Extensive experiments on 10 datasets demonstrate that our KCPT improves accuracy by 1–3% and stability by 26–40% on average compared to SOTA prompt tuning methods in low-resource settings.