<p>The deployment of Agentic AI creates governance challenges that existing frameworks cannot adequately address. This paper argues that when tasks are delegated to AI agents, a principal-agent relationship is established that is characterized by asymmetries, including hidden information, moral hazard, and adverse selection. The training pipeline of large language model-based Agentic AI produces these asymmetries: The pretraining process optimizes prediction over truth. The reinforcement learning process optimizes preference proxies, not factuality. The use of benchmarks to showcase the power of large language models does not help to assess if the model performs well on real-world data. The absence of “skin in the game” by the AI agent could lead to risk-seeking actions that a human agent would not consider. This makes AI agents susceptible to moral hazard. Therefore, we propose a dual-layer architecture that addresses the problem concerning the relation between the reliability and the accountability of Agentic AI systems. The Agent Layer is responsible for execution. The Governor Layer comprises four components: a Selection Gateway for ex ante screening, a Monitoring Engine for observability, Structural Constraints for action restrictions, and an Accountability Ledger that assigns responsibilities to stakeholders who could prevent them.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

No skin in the game: why agentic AI requires principal-agent governance

  • Martin Prause

摘要

The deployment of Agentic AI creates governance challenges that existing frameworks cannot adequately address. This paper argues that when tasks are delegated to AI agents, a principal-agent relationship is established that is characterized by asymmetries, including hidden information, moral hazard, and adverse selection. The training pipeline of large language model-based Agentic AI produces these asymmetries: The pretraining process optimizes prediction over truth. The reinforcement learning process optimizes preference proxies, not factuality. The use of benchmarks to showcase the power of large language models does not help to assess if the model performs well on real-world data. The absence of “skin in the game” by the AI agent could lead to risk-seeking actions that a human agent would not consider. This makes AI agents susceptible to moral hazard. Therefore, we propose a dual-layer architecture that addresses the problem concerning the relation between the reliability and the accountability of Agentic AI systems. The Agent Layer is responsible for execution. The Governor Layer comprises four components: a Selection Gateway for ex ante screening, a Monitoring Engine for observability, Structural Constraints for action restrictions, and an Accountability Ledger that assigns responsibilities to stakeholders who could prevent them.