Scholarlink

<p>Patient’s unique information needs about their hospitalization can be addressed using clinical evidence from electronic health records (EHRs) and artificial intelligence (AI). However, robust datasets to assess the factuality and relevance of AI-generated responses are lacking and, to our knowledge, none capture patient information needs in the context of their EHRs. To address this gap, we introduce ArchEHR-QA, an expert-annotated dataset of 134 cases from intensive care unit and emergency department settings. Cases comprise patient questions from public health forums, clinician-interpreted versions, relevant clinical note excerpts with sentence-level relevance annotations, and clinician-authored answers. To establish benchmarks for grounded EHR question answering (QA), we evaluated three open-weight large language models (Llama 4, Llama 3, and Mixtral) across three prompting strategies. We assessed performance on two dimensions: Factuality (overlap between cited and ground truth evidence) and Relevance (similarity to reference answers).</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Dataset for Addressing Patient’s Information Needs related to Clinical Course of Hospitalization

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月25日

查看全文
  • Sarvesh Soni,
  • Dina Demner-Fushman

摘要

Patient’s unique information needs about their hospitalization can be addressed using clinical evidence from electronic health records (EHRs) and artificial intelligence (AI). However, robust datasets to assess the factuality and relevance of AI-generated responses are lacking and, to our knowledge, none capture patient information needs in the context of their EHRs. To address this gap, we introduce ArchEHR-QA, an expert-annotated dataset of 134 cases from intensive care unit and emergency department settings. Cases comprise patient questions from public health forums, clinician-interpreted versions, relevant clinical note excerpts with sentence-level relevance annotations, and clinician-authored answers. To establish benchmarks for grounded EHR question answering (QA), we evaluated three open-weight large language models (Llama 4, Llama 3, and Mixtral) across three prompting strategies. We assessed performance on two dimensions: Factuality (overlap between cited and ground truth evidence) and Relevance (similarity to reference answers).