Scholarlink

<p>While AI ethics ensures fairness, accountability, and protection of user rights, dark patterns manipulate users to take unintended actions on digital interfaces. Related studies uncover limited insights into how reliably; human experts and AI models can detect dark patterns within a specific taxonomy. Our research fills this gap by asymmetrically examining cross-origin detection performance of human and AI/LLM evaluators (each evaluator’s ability to detect dark patterns generated by the opposite source) to understand their limitations and future potentials. Using GPT-4.1, we generated 200 UI images (with matched dark and non-dark pattern pairs) and selected 200 UI images collected 200 human-created UI screenshots from the ContextDP/AidUI dataset, based on computational, methodological, and statistical considerations. We calculated inter-rater reliability, recall, and error distribution. The results show that UX experts achieved substantial agreement (k = 0.75) and significantly higher recall (r = 0.99) over AI/LLMs. We present a novel study which explore the performance of AI/LLMs and UX experts in detecting dark patterns in UI images, and provide a benchmark dataset that could be useful to future research, while discussing empirical insights into the role, limitations, and promise of AI/LLMs in UI/UX design ethics and auditing, in realistic deployment scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

UX experts vs. AI: exploring the performance of large language models and humans on detecting dark patterns

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月5日

查看全文
  • Joshua Nwokeji,
  • Makuochi Nkwo,
  • Tochukwu Ikwunne,
  • Meiyeer Yeerbo

摘要

While AI ethics ensures fairness, accountability, and protection of user rights, dark patterns manipulate users to take unintended actions on digital interfaces. Related studies uncover limited insights into how reliably; human experts and AI models can detect dark patterns within a specific taxonomy. Our research fills this gap by asymmetrically examining cross-origin detection performance of human and AI/LLM evaluators (each evaluator’s ability to detect dark patterns generated by the opposite source) to understand their limitations and future potentials. Using GPT-4.1, we generated 200 UI images (with matched dark and non-dark pattern pairs) and selected 200 UI images collected 200 human-created UI screenshots from the ContextDP/AidUI dataset, based on computational, methodological, and statistical considerations. We calculated inter-rater reliability, recall, and error distribution. The results show that UX experts achieved substantial agreement (k = 0.75) and significantly higher recall (r = 0.99) over AI/LLMs. We present a novel study which explore the performance of AI/LLMs and UX experts in detecting dark patterns in UI images, and provide a benchmark dataset that could be useful to future research, while discussing empirical insights into the role, limitations, and promise of AI/LLMs in UI/UX design ethics and auditing, in realistic deployment scenarios.