Scholarlink

<p>Large Language Models have recently been applied to text annotation tasks from social sciences, equating or surpassing the performance of human workers at a fraction of the cost. However, very few inquiries in the social sciences have been made of the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at <a href="https://prompt-ultra.github.io/">https://prompt-ultra.github.io/</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Prompt selection matters: enhancing text annotations for social sciences with large language models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月11日

查看全文
  • Louis Abraham,
  • Charles Arnal,
  • Antoine Marie

摘要

Large Language Models have recently been applied to text annotation tasks from social sciences, equating or surpassing the performance of human workers at a fraction of the cost. However, very few inquiries in the social sciences have been made of the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at https://prompt-ultra.github.io/.