<p>Artificial intelligence (AI) chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. Although some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence and delusional thinking linked to emotional relationships with chatbots. To understand these risks, we need to consider the interaction between human cognitive-emotional biases and chatbot behavioral tendencies, the latter including companionship-reinforcing behaviors such as sycophancy, role play and anthropomimesis. Individuals with preexisting mental health conditions may face increased risks of chatbot-induced changes in beliefs and behavior, particularly where these conditions manifest in altered belief-updating, reality-testing and social isolation. To address this emerging public health concern, we need coordinated action across clinical practice, AI development and regulatory frameworks.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Technological folie à deux: feedback loops between AI chatbots and mental health

  • Sebastian Dohnány,
  • Zeb Kurth-Nelson,
  • Eleanor Spens,
  • Lennart Luettgau,
  • Alastair Reid,
  • Iason Gabriel,
  • Christopher Summerfield,
  • Murray Shanahan,
  • Matthew M. Nour

摘要

Artificial intelligence (AI) chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. Although some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence and delusional thinking linked to emotional relationships with chatbots. To understand these risks, we need to consider the interaction between human cognitive-emotional biases and chatbot behavioral tendencies, the latter including companionship-reinforcing behaviors such as sycophancy, role play and anthropomimesis. Individuals with preexisting mental health conditions may face increased risks of chatbot-induced changes in beliefs and behavior, particularly where these conditions manifest in altered belief-updating, reality-testing and social isolation. To address this emerging public health concern, we need coordinated action across clinical practice, AI development and regulatory frameworks.