The chapter addresses the emerging challenge posed by malicious actors using AI in support of the deliberate manipulation of human understanding, decision-making and ability to co-ordinate collective action. It describes how the development and testing of effective AI capabilities which can understand and manipulate the complex adaptive process of collective human decision-making will require the co-ordinated observation of online activity, activity in the physical world, and human cognitive states. It shows that technologically advanced authoritarian surveillance states are the actors best placed to develop such capabilities. This will provide them with the capability to control their society’s epistemic structures, develop information manipulation capabilities that others can only observe from the outside, and achieve instrumentarian power. Such a state when acting as a malicious actor has the potential to significantly over match other group’s epistemic security and to significantly manipulate group decision-making. In addition, the potential for future epistemic insecurity should be considered a priority global catastrophic risk.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Artificial Intelligence and the Malicious Manipulation of Human Decision-Making

  • Gavin Pearson

摘要

The chapter addresses the emerging challenge posed by malicious actors using AI in support of the deliberate manipulation of human understanding, decision-making and ability to co-ordinate collective action. It describes how the development and testing of effective AI capabilities which can understand and manipulate the complex adaptive process of collective human decision-making will require the co-ordinated observation of online activity, activity in the physical world, and human cognitive states. It shows that technologically advanced authoritarian surveillance states are the actors best placed to develop such capabilities. This will provide them with the capability to control their society’s epistemic structures, develop information manipulation capabilities that others can only observe from the outside, and achieve instrumentarian power. Such a state when acting as a malicious actor has the potential to significantly over match other group’s epistemic security and to significantly manipulate group decision-making. In addition, the potential for future epistemic insecurity should be considered a priority global catastrophic risk.