Re-thinking human–machine interaction and the governance of AI in the military domain
摘要
Human control is a key principle for the governance of artificial intelligence (AI) in the military domain, underlining the importance of human responsibility in the use of military AI. But what human control exactly entails is unclear. This Perspective proposes that safeguarding human control requires examining the diverse forms of human–machine interaction that occur across the AI life cycle. Here we review how actions at distinct stages, including research and development as well as testing, evaluation, validation and verification, impact human control. We evaluate these dynamics through potential applications of AI-based decision support systems to assist in international humanitarian law assessments. Arguing that dynamics of human–machine interaction require more attention to uphold human control, we offer three recommendations to bolster this principle: contestation mechanisms to help humans cross-check and validate information generated by AI; continuous training to help users navigate unexpected scenarios and contexts that lack sufficient data; and documentation.