Scholarlink

<p>In-context learning (ICL) possesses the remarkable ability to ignite the reasoning capabilities of large language models (LLMs) with mere handfuls of samples; yet its efficacy hinges heavily on the quality of demonstrations. Consequently, several approaches have been devised to bolster the performance of ICL through sophisticated demonstration retrieval techniques. However, in out-of-distribution (OOD) scenarios, even the most advanced retrieval strategies encounter formidable hurdles, as it is arduous to extract pertinent test-related knowledge from disparate demonstrations. To address these challenges, this paper introduces a novel context-aware retrieval framework that effectively mitigates the adverse effects of OOD discrepancies on various tasks. This framework leverages LLMs to generate domain-pertinent data and retrieves demonstrations from the newly generated corpus. Given the inherent challenge of acquiring labeled data in real-world applications, we further propose an innovative retrieval method that meticulously balances sample confidence, similarity, and diversity, which ensures the judicious utilization of unlabeled samples generated by LLM. Extensive practical experiments conducted on multiple LLMs and within the realm of natural language processing have unequivocally validated the OOD robustness of our proposed framework. Our code is available at <a href="https://github.com/songruiecho/Ralood">https://github.com/songruiecho/Ralood</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Retrieval augmentation for out-of-distribution robustness in non-knowledge intensive in-context learning

  • 期刊论文

  • 发表日期：

    2026年1月12日

查看全文
  • Rui Song,
  • Yingji Li,
  • Fausto Giunchiglia,
  • Hao Xu

摘要

In-context learning (ICL) possesses the remarkable ability to ignite the reasoning capabilities of large language models (LLMs) with mere handfuls of samples; yet its efficacy hinges heavily on the quality of demonstrations. Consequently, several approaches have been devised to bolster the performance of ICL through sophisticated demonstration retrieval techniques. However, in out-of-distribution (OOD) scenarios, even the most advanced retrieval strategies encounter formidable hurdles, as it is arduous to extract pertinent test-related knowledge from disparate demonstrations. To address these challenges, this paper introduces a novel context-aware retrieval framework that effectively mitigates the adverse effects of OOD discrepancies on various tasks. This framework leverages LLMs to generate domain-pertinent data and retrieves demonstrations from the newly generated corpus. Given the inherent challenge of acquiring labeled data in real-world applications, we further propose an innovative retrieval method that meticulously balances sample confidence, similarity, and diversity, which ensures the judicious utilization of unlabeled samples generated by LLM. Extensive practical experiments conducted on multiple LLMs and within the realm of natural language processing have unequivocally validated the OOD robustness of our proposed framework. Our code is available at https://github.com/songruiecho/Ralood.