Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as emergency response, medical triage, and security operations. This study uses a controlled, fictitious multiple casualty triage scenario to investigate the effects of position bias, the tendency of LLMs to prioritize information elements based on their position in a list rather than their relevance. For a list of patients, we vary the position of the most critically injured one to evaluate whether systematic deviation from the medically established START triage protocol can be observed for GPT-4o and GPT-4o mini. Our results reveal a consistent recency bias with both models: The most critically injured patient was less likely to be prioritized when listed first. This effect was more pronounced in shorter patient lists, challenging the common assumption that short prompts are less prone to evoke position bias. These findings raise critical concerns about the operational reliability of LLMs in time-sensitive, high-stakes tasks with high information density. Our study contributes to growing evidence that LLMs require rigorous validation before deployment in sensitive environments such as OSINT, defense informatics, and emergency triage.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Position Bias in LLMs for Critical Decision Support: A Case Study on Multiple Casualty Triage

  • Ulrika Wickenberg-Bolin,
  • Katie Cohen,
  • Helena Björnesjö,
  • Agnes Tegen

摘要

Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as emergency response, medical triage, and security operations. This study uses a controlled, fictitious multiple casualty triage scenario to investigate the effects of position bias, the tendency of LLMs to prioritize information elements based on their position in a list rather than their relevance. For a list of patients, we vary the position of the most critically injured one to evaluate whether systematic deviation from the medically established START triage protocol can be observed for GPT-4o and GPT-4o mini. Our results reveal a consistent recency bias with both models: The most critically injured patient was less likely to be prioritized when listed first. This effect was more pronounced in shorter patient lists, challenging the common assumption that short prompts are less prone to evoke position bias. These findings raise critical concerns about the operational reliability of LLMs in time-sensitive, high-stakes tasks with high information density. Our study contributes to growing evidence that LLMs require rigorous validation before deployment in sensitive environments such as OSINT, defense informatics, and emergency triage.