Scholarlink

<p>The purpose of this study is to assess the CALM-RAD framework that converts GPT-4o token-level log-probabilities (TLPs) into calibrated confidence scores for selective auto-labeling of TNM stage and primary site in head and neck cancer CT reports. Anonymized 150 CT reports were retrospectively curated from a tertiary cancer center. A radiologist assigned T, N, M, and site labels, which were hidden from the model. GPT-4o was queried once per report using simple and structured (knowledge-guided) prompts (temperature 0.2), returning single-token predictions and TLPs. Reports were split into calibration (<i>n</i> = 50) and testing (<i>n</i> = 100) sets. TLPs were converted to calibrated confidence scores using isotonic regression (M-stage excluded). Confidence-based triage was simulated on the test set to evaluate accuracy–coverage trade-offs. Correct predictions showed higher TLPs than errors (e.g., N-stage <i>U</i> = 1302.5, <i>P</i> &lt; 0.001). Simple prompt accuracies were 0.73 for T, 0.80 for N, and 0.96 for site; structured prompts raised T to 0.81 and N to 0.83, but reduced site to 0.91. Triage thresholds accepting 37% (simple) and 34% (structured) of T-stage predictions raised accuracies to 0.89 and 1.00, respectively; accepting 55% of N-stage cases achieved up to 1.00 accuracy. Accepted predictions were enriched for prototypical labels (e.g., N0 = 91%, <i>χ</i><sup>2</sup> = 79.9, <i>P</i> &lt; 0.001). CALM‑RAD demonstrates that calibrated log probabilities can support trustworthy selective automation in radiology report labeling while deferring uncertain cases to human review. This framework offers a practical path toward safer LLM integration in clinical radiology workflows.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CALM-RAD: Calibrated GPT-4o Confidence-Based Triage Enables 96–100% Accuracy in Automated TNM Staging of Head and Neck Cancer Reports

  • 期刊论文

  • 发表日期：

    2026年3月12日

查看全文
  • Amit Gupta,
  • Lisa C. Adams,
  • Krithika Rangarajan

摘要

The purpose of this study is to assess the CALM-RAD framework that converts GPT-4o token-level log-probabilities (TLPs) into calibrated confidence scores for selective auto-labeling of TNM stage and primary site in head and neck cancer CT reports. Anonymized 150 CT reports were retrospectively curated from a tertiary cancer center. A radiologist assigned T, N, M, and site labels, which were hidden from the model. GPT-4o was queried once per report using simple and structured (knowledge-guided) prompts (temperature 0.2), returning single-token predictions and TLPs. Reports were split into calibration (n = 50) and testing (n = 100) sets. TLPs were converted to calibrated confidence scores using isotonic regression (M-stage excluded). Confidence-based triage was simulated on the test set to evaluate accuracy–coverage trade-offs. Correct predictions showed higher TLPs than errors (e.g., N-stage U = 1302.5, P < 0.001). Simple prompt accuracies were 0.73 for T, 0.80 for N, and 0.96 for site; structured prompts raised T to 0.81 and N to 0.83, but reduced site to 0.91. Triage thresholds accepting 37% (simple) and 34% (structured) of T-stage predictions raised accuracies to 0.89 and 1.00, respectively; accepting 55% of N-stage cases achieved up to 1.00 accuracy. Accepted predictions were enriched for prototypical labels (e.g., N0 = 91%, χ2 = 79.9, P < 0.001). CALM‑RAD demonstrates that calibrated log probabilities can support trustworthy selective automation in radiology report labeling while deferring uncertain cases to human review. This framework offers a practical path toward safer LLM integration in clinical radiology workflows.