Scholarlink

<p>Most ambient AI medical scribes process audio only, omitting clinically important visual details. We developed a vision-enabled AI scribe using Google’s Gemini model and Ray-Ban Meta smart glasses to document medication histories—a task requiring both audio and visual input. Ten clinical pharmacists video-recorded 110 simulated medication history interviews. Following iterative prompt engineering on 10 training recordings, the scribe was evaluated on 100 test recordings (2160 data points) across patient details and medication-specific fields. The vision-enabled scribe achieved 98% overall accuracy (2114/2,160 data points), ranging from 96% for patient details to 99% for dosing directions and indication. Video input significantly outperformed audio-only processing (98% vs 81%, <i>P</i> &lt; 0.001), primarily through reduced omissions (10 vs 358 errors). Vision-enabled AI scribes substantially improved documentation accuracy for tasks requiring visual input, demonstrating potential to markedly reduce omission errors in clinical documentation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Vision-Enabled AI scribes reduce omissions in clinical conversations: evidence from simulated medication histories

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月26日

查看全文
  • Bradley D. Menz,
  • Nicholas L. Scarfo,
  • Natansh D. Modi,
  • Erik Cornelisse,
  • Lee X. Li,
  • Jin Quan Eugene Tan,
  • Jimit Gandhi,
  • Dorsa Maher,
  • Dib Kousa,
  • Kezia Daniel,
  • Vidya Menon,
  • Stephen Bacchi,
  • Ross A. McKinnon,
  • Michael D. Wiese,
  • Andrew Rowland,
  • Michael J. Sorich,
  • Ashley M. Hopkins

摘要

Most ambient AI medical scribes process audio only, omitting clinically important visual details. We developed a vision-enabled AI scribe using Google’s Gemini model and Ray-Ban Meta smart glasses to document medication histories—a task requiring both audio and visual input. Ten clinical pharmacists video-recorded 110 simulated medication history interviews. Following iterative prompt engineering on 10 training recordings, the scribe was evaluated on 100 test recordings (2160 data points) across patient details and medication-specific fields. The vision-enabled scribe achieved 98% overall accuracy (2114/2,160 data points), ranging from 96% for patient details to 99% for dosing directions and indication. Video input significantly outperformed audio-only processing (98% vs 81%, P < 0.001), primarily through reduced omissions (10 vs 358 errors). Vision-enabled AI scribes substantially improved documentation accuracy for tasks requiring visual input, demonstrating potential to markedly reduce omission errors in clinical documentation.