<p>This paper examines when it is impermissible to outsource bureaucratic decisions to machine learning systems. It distinguishes between institutional legitimacy and decision permissibility. The former refers to the authority of an institution whose decisions have impact on matters of public interest. The latter refers to the internal features of automated decision-making procedures that make it permissible to substitute organizational decision-making with automated processes. The paper argues that the violation of certain criteria makes it impermissible to outsource a decision to a machine learning system. These criteria include five necessary, but not sufficient, conditions: due appreciation, consistency, justifiability, accuracy, and contestability. Failing any of these conditions renders a machine learning decision impermissible. Lastly, it explores the possibility that meeting all conditions might still not suffice in more complex scenarios. </p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Five Boundaries of Permissible Automation

  • Diana Acosta-Navas,
  • Henrik D. Kugelberg

摘要

This paper examines when it is impermissible to outsource bureaucratic decisions to machine learning systems. It distinguishes between institutional legitimacy and decision permissibility. The former refers to the authority of an institution whose decisions have impact on matters of public interest. The latter refers to the internal features of automated decision-making procedures that make it permissible to substitute organizational decision-making with automated processes. The paper argues that the violation of certain criteria makes it impermissible to outsource a decision to a machine learning system. These criteria include five necessary, but not sufficient, conditions: due appreciation, consistency, justifiability, accuracy, and contestability. Failing any of these conditions renders a machine learning decision impermissible. Lastly, it explores the possibility that meeting all conditions might still not suffice in more complex scenarios.