Scholarlink

Purpose <p>Large Language Models (LLMs) extracts structured data from unstructured Ga-68 Prostate Specific Membrane Antigen Positron Emission Tomography/Computed Tomography (PSMA PET/CT) reports by interpreting context and semantics, enabling reliable conversion of free-text into structured, queryable data. We evaluated whether locally deployed Llama 3.2:3b and Gemma 2:2b models could accurately perform zero-shot extraction of key diagnostic features.</p> Methods <p>We retrospectively selected 50 de-identified Ga-68 PSMA PET/CT reports of histologically confirmed prostate cancer (01 January 2020–30 June 2024), ensuring rigorous expert annotation and inter-reader agreement. Reports were batch-processed using standardized zero-shot and few-shot prompts applied to locally deployed Llama 3.2:3b and Gemma 2:2b models, with model outputs benchmarked against an adjudicated dual-expert ground truth reference derived from independent annotations and consensus resolution of discrepancies using standard performance metrics.</p> Results <p>Both Llama 3.2:3b and Gemma 2:2b achieved rapid, reliable extraction of key diagnostic features from Ga-68 PSMA PET/CT reports, with excellent inter-reader agreement (average κ = 0.882). Llama 3.2:3b offered superior sensitivity (83.7%) and negative predictive value (NPV) (97.3%), while Gemma 2:2b demonstrated greater overall accuracy (86.2%) and specificity (88.5%). Model performance varied with question prevalence and complexity, highlighting distinct strengths for clinical integration.</p> Conclusion <p>Open-weight Llama 3.2:3b and Gemma 2:2b LLMs enable rapid extraction of key PET/CT findings (16-20x faster than manual review), with Llama excelling in sensitivity and Gemma in specificity. However, modest performance on rare but clinically important findings and limited positive predictive value indicate these models are best suited for initial screening or human-assisted workflows rather than autonomous extraction.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating Locally Deployed Large Language Models for Ga-68 PSMA PET/CT Report Mining in Prostate Cancer

  • 期刊论文

  • 发表日期：

    2026年3月3日

查看全文
  • Jagrati Chaudhary,
  • Param Dev Sharma,
  • Sanjay Kumar,
  • Nikita Takalkar,
  • Rakesh Kumar,
  • Kunhi Parambath Haresh,
  • Chandan J. Das,
  • Ranjit Kumar Sahoo,
  • Seema Kaushal,
  • M. Kalaivani,
  • Hemant Khairwa,
  • Anil Kumar Pandey

摘要

Purpose

Large Language Models (LLMs) extracts structured data from unstructured Ga-68 Prostate Specific Membrane Antigen Positron Emission Tomography/Computed Tomography (PSMA PET/CT) reports by interpreting context and semantics, enabling reliable conversion of free-text into structured, queryable data. We evaluated whether locally deployed Llama 3.2:3b and Gemma 2:2b models could accurately perform zero-shot extraction of key diagnostic features.

Methods

We retrospectively selected 50 de-identified Ga-68 PSMA PET/CT reports of histologically confirmed prostate cancer (01 January 2020–30 June 2024), ensuring rigorous expert annotation and inter-reader agreement. Reports were batch-processed using standardized zero-shot and few-shot prompts applied to locally deployed Llama 3.2:3b and Gemma 2:2b models, with model outputs benchmarked against an adjudicated dual-expert ground truth reference derived from independent annotations and consensus resolution of discrepancies using standard performance metrics.

Results

Both Llama 3.2:3b and Gemma 2:2b achieved rapid, reliable extraction of key diagnostic features from Ga-68 PSMA PET/CT reports, with excellent inter-reader agreement (average κ = 0.882). Llama 3.2:3b offered superior sensitivity (83.7%) and negative predictive value (NPV) (97.3%), while Gemma 2:2b demonstrated greater overall accuracy (86.2%) and specificity (88.5%). Model performance varied with question prevalence and complexity, highlighting distinct strengths for clinical integration.

Conclusion

Open-weight Llama 3.2:3b and Gemma 2:2b LLMs enable rapid extraction of key PET/CT findings (16-20x faster than manual review), with Llama excelling in sensitivity and Gemma in specificity. However, modest performance on rare but clinically important findings and limited positive predictive value indicate these models are best suited for initial screening or human-assisted workflows rather than autonomous extraction.