<p>Automated machine learning (AutoML) promises “easy AI” by lowering barriers to predictive modeling, but this convenience can mask serious deployment risks if leakage, weak validation, poor calibration, and subgroup performance are not rigorously examined. Existing responsible-AI frameworks provide valuable guidance, yet they typically do not serve as enforceable pre-deployment gates that translate governance principles into explicit ship/no-ship decisions. In this paper, we introduce the Automation Risk Checklist (ARC) as a practical deployment gate for AutoML pipelines and demonstrate its impact through a healthcare case study. ARC combines auditable checks across data integrity, leakage prevention, structure-aware validation, calibration, fairness, reproducibility, and operational readiness. We evaluate ARC on the Diabetes 130-US Hospitals dataset, a realistic setting for common AutoML risks because it contains repeated patient entries, post-discharge variables with leakage potential, and subgroup fields relevant to fairness assessment. In a naïve evaluation, models achieved apparent perfect discrimination (AUROC = 1.0). After enforcing ARC-based exclusions and patient-level grouped validation, performance fell to more realistic levels (AUROC approximately 0.64), revealing the extent of leakage-driven inflation. ARC also identified underconfident probability estimates in some models, with calibration improving after temperature scaling, and enabled transparent subgroup auditing across sex and race. This study does not claim universal validation of ARC across domains. Rather, it provides proof-of-impact that an auditable deployment gate can materially alter conclusions about AutoML readiness in a realistic, leakage-prone setting. ARC is intended as a practical governance mechanism to help teams document, test, and justify deployment decisions more transparently in high-stakes applications.</p>