Coercive risk and the lottery default
摘要
Public administrations use predictive systems to decide whom to investigate for fraud or for other violations. In this paper, I show that many of these systems distribute the risk of exposure to the state’s coercive powers. These systems are often assessed in terms of familiar notions of algorithmic fairness, and are treated as analogous to systems that simply allocate scarce goods. I argue that this way of looking at these tools is morally misleading, because coercion requires a more demanding justification. The paper uses Scanlonian contractualism and argues that unequal coercive risk is permissible only if it is justified by reasons that can be addressed to the particular person who bears that risk. These reasons are often absent in practice. Population-level correlations or expected gains in accuracy count only insofar as they can be personalized into individualist reasons. However, in the administrative contexts that I discuss in the paper, they will typically be insufficient to justify imposing greater coercive risk on some persons than on others. If there are no stronger admissible reasons to justify differential selection, the default should therefore be an equal-chance lottery. To strengthen the argument, I add two supporting claims. It is not evident that profiling can sustain deterrence better than lotteries, and lotteries are in some respects epistemically superior.