Scholarlink

This study evaluates and compares the effectiveness of rule-based methods and Large Language Models (LLMs) in detecting electronic protected health information (ePHI) within DICOM files, with a particular focus on Polish names and identifiers. The research utilized multiple test datasets containing DICOM and JSON data with identifiers and popular, rare Polish names, as well as false names, which do not exist. We compared several approaches: rule-based methods (STRICT and ANY mode), state-of-the-art (SOTA) models (GPT-4o, DeepSeek v.3, Bielik v.2, and Llama v3.3), and the RoBERTa De-ID Named Entity Recognition (NER) model. Results demonstrated that rule-based methods achieved high precision and recall for structured data, with near-perfect performance for popular and rare names. However, LLMs, while flexible, require high false positive rates and significant processing time. Rule-based algorithms are currently more effective for ePHI detection in DICOM attributes, but they struggle with free-text attribute values. Fine-tuning LLMs is suggested as a potential improvement for ePHI detection in medical imaging data.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Detection of Electronic Protected Health Information in DICOM Objects: Rule-Based Methods vs. LLMs

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Dmytro Tkachenko,
  • Milena Sobotka,
  • Antoni Górecki,
  • Jakub Kłopotek-Główczewski,
  • Tomasz Neumann,
  • Jacek Rumiński

摘要

This study evaluates and compares the effectiveness of rule-based methods and Large Language Models (LLMs) in detecting electronic protected health information (ePHI) within DICOM files, with a particular focus on Polish names and identifiers. The research utilized multiple test datasets containing DICOM and JSON data with identifiers and popular, rare Polish names, as well as false names, which do not exist. We compared several approaches: rule-based methods (STRICT and ANY mode), state-of-the-art (SOTA) models (GPT-4o, DeepSeek v.3, Bielik v.2, and Llama v3.3), and the RoBERTa De-ID Named Entity Recognition (NER) model. Results demonstrated that rule-based methods achieved high precision and recall for structured data, with near-perfect performance for popular and rare names. However, LLMs, while flexible, require high false positive rates and significant processing time. Rule-based algorithms are currently more effective for ePHI detection in DICOM attributes, but they struggle with free-text attribute values. Fine-tuning LLMs is suggested as a potential improvement for ePHI detection in medical imaging data.