Scholarlink

Background and Aims <p>Health literacy significantly impacts patient outcomes. While the average American reads at an 8th grade reading level, healthcare materials are often written above this, potentially contributing to our nation’s health literacy gap. Large language models (LLM) and prompt engineering may be able to help address this gap by consistently generating materials at the recommended 6th grade reading level. Our study aims to assess how different prompting techniques affect the readability of LLM-generated materials.</p> Methods <p>We assessed the effects of five prompt techniques (Zero-Shot, Contextualized, Constrained, Meta, Persona) on the readability of LLM-generated explanations for fifteen common gastroenterology and hepatology conditions across twelve LLMs. Output (<i>n</i> = 2655) readability was assessed with two readability metrics (Simple Measure of Gobbledygook (SMOG) index, Flesch-Kincaid Grade Level (FKGL)), followed by significance testing and post-hoc analysis.</p> Results <p>No prompt technique or model consistently produced outputs at or below a 6th grade reading level when assessed by the SMOG index, the preferred metric when assessing healthcare materials (<i>p</i> &lt; 0.001). However, prompts with less constraints yield less readable outputs, while prompts with more constraints yield significantly more readable outputs (<i>p</i> &lt; 0.001).</p> Conclusions <p>This study demonstrates the potential of LLMs as a tool in addressing America’s health literacy gap, as we show prompt engineering affects the readability of gastroenterology and hepatology-related explanations. We also found limitations to this technique. Further optimization is necessary before LLMs can consistently generate patient materials without appropriate clinician oversight, but it implies prompt engineering as a tool in addressing our nation’s health literacy gap.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

The Impact of Specific Prompt Engineering Techniques on the Readability of LLM-Generated Patient Materials in Gastroenterology and Hepatology

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月9日

查看全文
  • Husayn F. Ramji,
  • Aishwarya Gatiganti,
  • Anveet Janwadkar,
  • Jacob Lampenfeld,
  • Ilaria M. Simeone,
  • Abhijith Atkuru,
  • Jason Mathias,
  • Stephanie Mrowczynski,
  • Sharan Poonja,
  • Chandler Gilliard,
  • Shaquille Lewis,
  • Pooja Arumugam,
  • Clara Freedman,
  • Luis Morales,
  • Everette Martin III,
  • Larry Zhou,
  • Corinne Zalomek,
  • Devika Dixit,
  • Saba Abdulsada,
  • Matthew Houle,
  • Matthew Alias,
  • Molly Delk,
  • Sarah C. Glover,
  • Peng-Sheng Ting

摘要

Background and Aims

Health literacy significantly impacts patient outcomes. While the average American reads at an 8th grade reading level, healthcare materials are often written above this, potentially contributing to our nation’s health literacy gap. Large language models (LLM) and prompt engineering may be able to help address this gap by consistently generating materials at the recommended 6th grade reading level. Our study aims to assess how different prompting techniques affect the readability of LLM-generated materials.

Methods

We assessed the effects of five prompt techniques (Zero-Shot, Contextualized, Constrained, Meta, Persona) on the readability of LLM-generated explanations for fifteen common gastroenterology and hepatology conditions across twelve LLMs. Output (n = 2655) readability was assessed with two readability metrics (Simple Measure of Gobbledygook (SMOG) index, Flesch-Kincaid Grade Level (FKGL)), followed by significance testing and post-hoc analysis.

Results

No prompt technique or model consistently produced outputs at or below a 6th grade reading level when assessed by the SMOG index, the preferred metric when assessing healthcare materials (p < 0.001). However, prompts with less constraints yield less readable outputs, while prompts with more constraints yield significantly more readable outputs (p < 0.001).

Conclusions

This study demonstrates the potential of LLMs as a tool in addressing America’s health literacy gap, as we show prompt engineering affects the readability of gastroenterology and hepatology-related explanations. We also found limitations to this technique. Further optimization is necessary before LLMs can consistently generate patient materials without appropriate clinician oversight, but it implies prompt engineering as a tool in addressing our nation’s health literacy gap.