Scholarlink

Background <p>Large language Models (LLM) have demonstrated near human-level performance in medical domain, from summarizing clinical notes and passing medical licensing examinations, to predictive tasks such as disease diagnoses and treatment recommendations. However, currently there is little research on their efficacy for medical coding, a pivotal component in health informatics, clinical trials, and reimbursement management. This study proposes a prompt framework and investigates its effectiveness for medical coding.</p> Methods <p>First, a prompt-based learning framework is proposed. This framework aims to improve the performance of GPT for complex medical coding tasks by augmenting it with specific prior knowledge, utilizing techniques such as meta prompt, many-shot learning, and dynamic in-context learning. Then its effectiveness is evaluated with three different medical coding tasks. Finally, ablation studies are presented to validate and analyze the contribution of each module in the proposed prompt framework.</p> Results <p>For the MIMIC-IV dataset, the prediction accuracies of top-1 and top-5 for the 30 most frequent MS-DRG codes are 68.1% and 90.0%, respectively. The clinical trial criteria coding task results in a macro-F1 score of 68.4 on the CHIP-CTC test dataset in Chinese. Both results are comparable to the best methods in comparison that training deep leaning models or fine-tuning LLMs.</p> Conclusions <p>This study demonstrates that for targeted coding tasks, DRG and other simplified scenarios, off-the-shelf GPT models guided by carefully designed prompts can achieve performance comparable to state-of-the-art methods. While current GPT models serve as helpful assistants to human coding specialists,they are not yet equipped to fully replace expert judgment. Importantly, the recent and drastic decline of the cost of deploying large language models makes their integration into medical coding workflows increasingly feasible. As these models continue to evolve, their potential to provide reliable support for DRG coding task is likely to grow rapidly over time.</p> Trial registration <p>Not applicable.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Can LLMs effectively assist medical coding? Evaluating GPT performance on DRG and targeted clinical tasks

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年8月19日

查看全文
  • Yeli Feng

摘要

Background

Large language Models (LLM) have demonstrated near human-level performance in medical domain, from summarizing clinical notes and passing medical licensing examinations, to predictive tasks such as disease diagnoses and treatment recommendations. However, currently there is little research on their efficacy for medical coding, a pivotal component in health informatics, clinical trials, and reimbursement management. This study proposes a prompt framework and investigates its effectiveness for medical coding.

Methods

First, a prompt-based learning framework is proposed. This framework aims to improve the performance of GPT for complex medical coding tasks by augmenting it with specific prior knowledge, utilizing techniques such as meta prompt, many-shot learning, and dynamic in-context learning. Then its effectiveness is evaluated with three different medical coding tasks. Finally, ablation studies are presented to validate and analyze the contribution of each module in the proposed prompt framework.

Results

For the MIMIC-IV dataset, the prediction accuracies of top-1 and top-5 for the 30 most frequent MS-DRG codes are 68.1% and 90.0%, respectively. The clinical trial criteria coding task results in a macro-F1 score of 68.4 on the CHIP-CTC test dataset in Chinese. Both results are comparable to the best methods in comparison that training deep leaning models or fine-tuning LLMs.

Conclusions

This study demonstrates that for targeted coding tasks, DRG and other simplified scenarios, off-the-shelf GPT models guided by carefully designed prompts can achieve performance comparable to state-of-the-art methods. While current GPT models serve as helpful assistants to human coding specialists,they are not yet equipped to fully replace expert judgment. Importantly, the recent and drastic decline of the cost of deploying large language models makes their integration into medical coding workflows increasingly feasible. As these models continue to evolve, their potential to provide reliable support for DRG coding task is likely to grow rapidly over time.

Trial registration

Not applicable.