Scholarlink

Purpose <p>Mental health services worldwide face demand that consistently exceeds available capacity, leaving many people without timely access to support. Generative AI chat assistants offer a scalable way to help close this gap, but only if their conversational flexibility can be reconciled with the protocol fidelity, crisis safety, and auditability that safety-critical therapeutic use demands. This paper proposes the Bounded Generative Framework (BGF) as an architectural pattern for deploying large language models (LLMs) in such contexts, using an Emotional Freedom Techniques (EFT) chat intervention as a case study. It addresses this bounded generativity problem: retaining empathic, adaptive responses while enforcing protocol fidelity, crisis safety, and auditable behaviour.</p> Methodology <p>We implemented the BGF in Tapaway, a web-based EFT application that integrates a two-layer pipeline, a 10-state therapeutic protocol controller plus a terminal crisis state, and structured tool calls for JSON-based directives. We compared three LLMs (GPT-4o, Claude 3 Haiku, and Gemini 2.5 Flash). Technical performance was evaluated on a 29-case synthetic benchmark spanning standard, edge, adversarial, crisis, and protocol-compliance scenarios. Clinical quality was explored using authentic EFT sessions. LLM security was examined via OWASP-aligned adversarial tests targeting prompt injection, unsafe output handling, and data leakage.</p> Results <p>Across synthetic cases, models achieved high JSON parse success and directive validity, with GPT-4o yielding the strongest protocol fidelity and crisis-intent performance at the cost of higher latency. In application, paired sessions in this uncontrolled convenience sample of university students showed large within-session reductions in self-reported distress (Cohen’s d = 1.85), which should be interpreted as a descriptive feasibility signal rather than evidence of efficacy. Crisis detection yielded no false negatives in this sample (sensitivity 100%, 95% CI 69.2–100% across <i>N</i> = 10 combined crisis-positive cases), though these preliminary findings require validation in adequately powered studies, and security testing indicated partial robustness to common LLM-specific risks.</p> Conclusion <p>The BGF shows that safety-bounded, protocol-controlled generative interventions are feasible for EFT chat delivery, shifting assurance from model-only alignment towards application-level constraints. While promising as a design pattern for therapeutic AI, the framework does not establish clinical equivalence to human-delivered therapy; controlled trials are required to assess comparative and durable outcomes.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Designing a safe generative AI-powered chat assistance system for mental health using a bounded generative framework

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月20日

查看全文
  • Udochukwu John-bright Okike,
  • Amarachi Tyndale Eche,
  • Chimeremeze Victor Amaechi,
  • Pradeep Hewage,
  • Anchal Garg,
  • Adel Houten

摘要

Purpose

Mental health services worldwide face demand that consistently exceeds available capacity, leaving many people without timely access to support. Generative AI chat assistants offer a scalable way to help close this gap, but only if their conversational flexibility can be reconciled with the protocol fidelity, crisis safety, and auditability that safety-critical therapeutic use demands. This paper proposes the Bounded Generative Framework (BGF) as an architectural pattern for deploying large language models (LLMs) in such contexts, using an Emotional Freedom Techniques (EFT) chat intervention as a case study. It addresses this bounded generativity problem: retaining empathic, adaptive responses while enforcing protocol fidelity, crisis safety, and auditable behaviour.

Methodology

We implemented the BGF in Tapaway, a web-based EFT application that integrates a two-layer pipeline, a 10-state therapeutic protocol controller plus a terminal crisis state, and structured tool calls for JSON-based directives. We compared three LLMs (GPT-4o, Claude 3 Haiku, and Gemini 2.5 Flash). Technical performance was evaluated on a 29-case synthetic benchmark spanning standard, edge, adversarial, crisis, and protocol-compliance scenarios. Clinical quality was explored using authentic EFT sessions. LLM security was examined via OWASP-aligned adversarial tests targeting prompt injection, unsafe output handling, and data leakage.

Results

Across synthetic cases, models achieved high JSON parse success and directive validity, with GPT-4o yielding the strongest protocol fidelity and crisis-intent performance at the cost of higher latency. In application, paired sessions in this uncontrolled convenience sample of university students showed large within-session reductions in self-reported distress (Cohen’s d = 1.85), which should be interpreted as a descriptive feasibility signal rather than evidence of efficacy. Crisis detection yielded no false negatives in this sample (sensitivity 100%, 95% CI 69.2–100% across N = 10 combined crisis-positive cases), though these preliminary findings require validation in adequately powered studies, and security testing indicated partial robustness to common LLM-specific risks.

Conclusion

The BGF shows that safety-bounded, protocol-controlled generative interventions are feasible for EFT chat delivery, shifting assurance from model-only alignment towards application-level constraints. While promising as a design pattern for therapeutic AI, the framework does not establish clinical equivalence to human-delivered therapy; controlled trials are required to assess comparative and durable outcomes.