Scholarlink

Introduction <p>Accurate preoperative staging is vital in bladder cancer management, particularly for assessing muscle invasion. Multiparametric MRI (mpMRI) combined with the Vesical Imaging–Reporting and Data System (VI-RADS) offers a non-invasive and standardized approach for tumor stratification. However, inter-observer variability among radiologists remains a significant limitation. This study investigates the performance of three large language models (LLMs)—ChatGPT (OpenAI; GPT-5.2), Gemini (Google; Gemini 2.0), and Copilot (Microsoft; Copilot based on GPT-5 architecture)—in classifying bladder lesions according to VI-RADS using zero-shot and few-shot prompting strategies.</p> Materials and Methods <p>A synthetic dataset of 100 simulated bladder cancer cases was developed from expert-crafted synthetic radiology reports based on structured text descriptors and expert consensus, comprising 20 cases for each VI-RADS category (1–5). Each case included structured imaging descriptors aligned with VI-RADS scoring criteria. The LLMs were evaluated under zero-shot (no examples provided) and few-shot (with illustrative examples) prompting conditions. Performance metrics included accuracy, macro F1 scores, and Cohen’s kappa, with statistical significance assessed using McNemar’s test.</p> Results <p>Few-shot prompting significantly improved the classification performance of ChatGPT (OpenAI; GPT-5.2) (accuracy: 94%, F1: 0.939) and Gemini (Google; Gemini 2.0) (accuracy: 83%, F1: 0.823) compared to zero-shot. In contrast, Copilot (Microsoft; Copilot based on GPT-5 architecture)‘s accuracy (53%) and F1 score (0.503) declined under few-shot conditions. ChatGPT (OpenAI; GPT-5.2) demonstrated the highest consistency in identifying high-risk lesions (VI-RADS 4–5), followed by Gemini (Google; Gemini 2.0).</p> Conclusion <p>Few-shot prompting enhances LLM performance in VI-RADS classification, particularly for ChatGPT (OpenAI; GPT-5.2) and Gemini (Google; Gemini 2.0). These findings highlight the potential of AI tools to support radiological decision-making in bladder cancer staging. Further studies using real imaging data and explainable AI are warranted.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluation of large language models for VI-RADS reports: a comparative analysis of zero-shot and few-shot prompting

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月11日

查看全文
  • Ahmet Halis,
  • Deniz Celiker

摘要

Introduction

Accurate preoperative staging is vital in bladder cancer management, particularly for assessing muscle invasion. Multiparametric MRI (mpMRI) combined with the Vesical Imaging–Reporting and Data System (VI-RADS) offers a non-invasive and standardized approach for tumor stratification. However, inter-observer variability among radiologists remains a significant limitation. This study investigates the performance of three large language models (LLMs)—ChatGPT (OpenAI; GPT-5.2), Gemini (Google; Gemini 2.0), and Copilot (Microsoft; Copilot based on GPT-5 architecture)—in classifying bladder lesions according to VI-RADS using zero-shot and few-shot prompting strategies.

Materials and Methods

A synthetic dataset of 100 simulated bladder cancer cases was developed from expert-crafted synthetic radiology reports based on structured text descriptors and expert consensus, comprising 20 cases for each VI-RADS category (1–5). Each case included structured imaging descriptors aligned with VI-RADS scoring criteria. The LLMs were evaluated under zero-shot (no examples provided) and few-shot (with illustrative examples) prompting conditions. Performance metrics included accuracy, macro F1 scores, and Cohen’s kappa, with statistical significance assessed using McNemar’s test.

Results

Few-shot prompting significantly improved the classification performance of ChatGPT (OpenAI; GPT-5.2) (accuracy: 94%, F1: 0.939) and Gemini (Google; Gemini 2.0) (accuracy: 83%, F1: 0.823) compared to zero-shot. In contrast, Copilot (Microsoft; Copilot based on GPT-5 architecture)‘s accuracy (53%) and F1 score (0.503) declined under few-shot conditions. ChatGPT (OpenAI; GPT-5.2) demonstrated the highest consistency in identifying high-risk lesions (VI-RADS 4–5), followed by Gemini (Google; Gemini 2.0).

Conclusion

Few-shot prompting enhances LLM performance in VI-RADS classification, particularly for ChatGPT (OpenAI; GPT-5.2) and Gemini (Google; Gemini 2.0). These findings highlight the potential of AI tools to support radiological decision-making in bladder cancer staging. Further studies using real imaging data and explainable AI are warranted.