Scholarlink

<p>To evaluate and compare the performance of four Large Language Models (LLMs) in anesthesia decision-making for critically ill obstetric and geriatric patients and analyze their decision reliability across different surgical specialties. Prospective comparative analysis using standardized case evaluations. Four LLMs (ChatGPT-4o, Claude 3.5 Sonnet, DeepSeek-R1, and Grok 3). Thirty complex surgical cases (10 obstetric, 20 geriatric; 8 specialties) were analyzed. A 12-dimensional framework tested the models using unified prompts and decision points. Five trained anesthesiologists independently evaluated the models across six dimensions (patient assessment, anesthesia plan, risk management, individualization, contingency planning, decision logic; 1–10 scale, total 6–60). Overall, DeepSeek performed best (51.43 ± 2.74 points), significantly outperforming other models (<i>P</i> &lt; 0.001). For obstetric cases, the mean scores were: DeepSeek (52.00 ± 1.83), Grok (49.40 ± 3.06), ChatGPT (47.60 ± 2.88), and Claude (46.60 ± 2.17). For geriatric cases, scores were: DeepSeek (51.15 ± 3.10), Grok (48.60 ± 2.33), ChatGPT (47.35 ± 2.50), and Claude (45.75 ± 2.05). Across specialties, all models performed best in hepatobiliary surgery, burn surgery, and thoracic surgery. DeepSeek demonstrated consistent performance across all dimensions, with notable advantages in decision logic (8.80 ± 0.40) and contingency planning (8.27 ± 0.45). All LLMs demonstrated strong anesthesia decision-making capabilities, with DeepSeek showing the best overall performance. Exploratory analysis revealed performance variations across specialties, although small sample sizes preclude definitive conclusions. Clinical implementation should consider specialty-specific factors and decision process characteristics.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Performance of Large Language Models in Complex Anesthesia Decision-Making: A Comparative Study of Four LLMs in High-Risk Patients

  • 期刊论文

  • 发表日期：

    2025年10月1日

查看全文
  • Qian Ruan,
  • Jinghong Shi,
  • Yunke Dai,
  • Pingliang Yang,
  • Na Zhu,
  • Shun Wang

摘要

To evaluate and compare the performance of four Large Language Models (LLMs) in anesthesia decision-making for critically ill obstetric and geriatric patients and analyze their decision reliability across different surgical specialties. Prospective comparative analysis using standardized case evaluations. Four LLMs (ChatGPT-4o, Claude 3.5 Sonnet, DeepSeek-R1, and Grok 3). Thirty complex surgical cases (10 obstetric, 20 geriatric; 8 specialties) were analyzed. A 12-dimensional framework tested the models using unified prompts and decision points. Five trained anesthesiologists independently evaluated the models across six dimensions (patient assessment, anesthesia plan, risk management, individualization, contingency planning, decision logic; 1–10 scale, total 6–60). Overall, DeepSeek performed best (51.43 ± 2.74 points), significantly outperforming other models (P < 0.001). For obstetric cases, the mean scores were: DeepSeek (52.00 ± 1.83), Grok (49.40 ± 3.06), ChatGPT (47.60 ± 2.88), and Claude (46.60 ± 2.17). For geriatric cases, scores were: DeepSeek (51.15 ± 3.10), Grok (48.60 ± 2.33), ChatGPT (47.35 ± 2.50), and Claude (45.75 ± 2.05). Across specialties, all models performed best in hepatobiliary surgery, burn surgery, and thoracic surgery. DeepSeek demonstrated consistent performance across all dimensions, with notable advantages in decision logic (8.80 ± 0.40) and contingency planning (8.27 ± 0.45). All LLMs demonstrated strong anesthesia decision-making capabilities, with DeepSeek showing the best overall performance. Exploratory analysis revealed performance variations across specialties, although small sample sizes preclude definitive conclusions. Clinical implementation should consider specialty-specific factors and decision process characteristics.