Through Learning Analytics, Artificial Intelligence (AI) is introduced in educational contexts with the purpose of reaching closed-loop interventions to help students who need academic or behavioral help. Although multiple predictive models have been trained based on available educational data, few implementations have been made due to the implications they have on learners and teachers. Keeping humans in the loop permits us to contextualize those predictions and recommendations given by an AI system to prevent unforeseen consequences. For instance, dropout predictions might become a self-fulfilling prophecy if they are not properly communicated or could even exclude underrepresented students. In the pursuit of autonomous interventions in educational contexts, we must consider how these AI systems will affect the lives of learners and teachers. We illustrate ethical dilemmas in AI systems used to prevent student dropout based on UNESCO’s Recommendation on the Ethics of AI and on the bias that might arise in data, models, and applications.