<p>Black-box AI cannot provide causal explanations for the decisions it makes, but medical AI has shown great promise as an accurate and reliable technology that both improves the quality of patient care and provides better access to healthcare for more patients. There is an ethical argument that to meet the informational requirements of patient autonomy, medical decision-making ought to be explainable to the patient. As such, there have been claims that black-box AI ought to be only minimally used in healthcare. This paper seeks to argue that black-box AI within the clinical context does not necessarily undermine patient autonomy, as defined in standard or relational accounts. Rather, patient autonomy is affected primarily by how AI tools are used. </p>