Scholarlink

<p>Language models today are trained to convey confidence in their outputs, regardless of whether those outputs are correct. The alignment methods we use to make them helpful also push them toward unwarranted certainty, rewarding decisive answers over appropriate hedging. As these foundation models enter high-stakes domains such as science and medicine, this disconnect between how sure they sound and how accurate they are can become dangerous. Here, we examine why post-training degrades a model’s sense of uncertainty, and we review techniques that can bring expressed confidence back in line with actual reliability. Through this, we argue that trustworthy AI means treating calibration as a core design goal.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A crisis of overconfidence: Why confidence, not accuracy, is the real risk in clinical AI

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月5日

查看全文
  • Jacob S. Berkowitz,
  • Jake R. Patock,
  • Asma Nawaz,
  • Graciela Gonzalez-Hernandez,
  • Nicholas P. Tatonetti

摘要

Language models today are trained to convey confidence in their outputs, regardless of whether those outputs are correct. The alignment methods we use to make them helpful also push them toward unwarranted certainty, rewarding decisive answers over appropriate hedging. As these foundation models enter high-stakes domains such as science and medicine, this disconnect between how sure they sound and how accurate they are can become dangerous. Here, we examine why post-training degrades a model’s sense of uncertainty, and we review techniques that can bring expressed confidence back in line with actual reliability. Through this, we argue that trustworthy AI means treating calibration as a core design goal.