Scholarlink

<p>The clinical adoption of artificial intelligence (AI) has focused on enabling automation, but conventional accuracy metrics fail to answer a key question: when is it safe to trust an AI system? We introduce the Safety-Aware Receiver Operating Characteristic (SA-ROC) framework, which defines operational safety as an ability to meet pre-specified reliability levels. The SA-ROC curve delineates a Rule-in and a Rule-out Safe Zone, where autonomous action is permitted, and a Gray Zone, where human review is mandated. To quantify this non-automated workload, we introduce the Gray Zone Area (Γ<sub>Area</sub>), a metric measuring the operational cost of indecision. Our framework reveals a key reversal: in a case study of two FDA-cleared algorithms for cancer screening, the model with a statistically superior AUC was found to be operationally less safe for high-confidence screening. SA-ROC enables active governance, translating clinical policy into optimized workflows that inform operational safety and complement regulatory safety evaluation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Defining operational safety in clinical artificial intelligence systems

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月20日

查看全文
  • Young-Tak Kim,
  • Hyunji Kim,
  • Manisha Bahl,
  • Michael H. Lev,
  • Ramon Gilberto González,
  • Michael S. Gee,
  • Synho Do

摘要

The clinical adoption of artificial intelligence (AI) has focused on enabling automation, but conventional accuracy metrics fail to answer a key question: when is it safe to trust an AI system? We introduce the Safety-Aware Receiver Operating Characteristic (SA-ROC) framework, which defines operational safety as an ability to meet pre-specified reliability levels. The SA-ROC curve delineates a Rule-in and a Rule-out Safe Zone, where autonomous action is permitted, and a Gray Zone, where human review is mandated. To quantify this non-automated workload, we introduce the Gray Zone Area (ΓArea), a metric measuring the operational cost of indecision. Our framework reveals a key reversal: in a case study of two FDA-cleared algorithms for cancer screening, the model with a statistically superior AUC was found to be operationally less safe for high-confidence screening. SA-ROC enables active governance, translating clinical policy into optimized workflows that inform operational safety and complement regulatory safety evaluation.