Scholarlink

<p>Artificial general intelligence (AGI) features prominently in some existential risk literature, according to which the development of AGI greatly increases possible AI-induced risks to humanity. But we argue that the typical concept of AGI may be ill-suited for conceptualizing those systems that pose the greatest risks. In particular, AGI does not account for how AI agents’ abilities and behavioral strategies could be affected by complex multi-agent environments. Accordingly, we develop a simple formal model for what we call <i>power-intelligence</i>, which assesses agents’ capacities to exert influence over a resemblance class of multi-agent environments. We then taxonomize and examine multi-agent relational dynamics including competition, dependence, and complementarity. This allows us to characterize a concept of <i>dangerous</i> AI, which conjoins high power-intelligence with multi-agent dynamics that might incentivize harms to other agents. We argue that this concept of dangerous AI is more informative for existential risk assessments than both typical conceptions of AGI and “power-seeking” intelligence in the literature.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Defining dangerous AI: existential risk, power-intelligence, and the limits of AGI

  • 期刊论文

  • 发表日期：

    2025年7月31日

查看全文
  • Reuben Sass

摘要

Artificial general intelligence (AGI) features prominently in some existential risk literature, according to which the development of AGI greatly increases possible AI-induced risks to humanity. But we argue that the typical concept of AGI may be ill-suited for conceptualizing those systems that pose the greatest risks. In particular, AGI does not account for how AI agents’ abilities and behavioral strategies could be affected by complex multi-agent environments. Accordingly, we develop a simple formal model for what we call power-intelligence, which assesses agents’ capacities to exert influence over a resemblance class of multi-agent environments. We then taxonomize and examine multi-agent relational dynamics including competition, dependence, and complementarity. This allows us to characterize a concept of dangerous AI, which conjoins high power-intelligence with multi-agent dynamics that might incentivize harms to other agents. We argue that this concept of dangerous AI is more informative for existential risk assessments than both typical conceptions of AGI and “power-seeking” intelligence in the literature.