Educational apps and platforms that utilize predictive analytics are becoming increasingly common in higher education. While these tools offer valuable insights into student learning and provide recommendations for educators, they also introduce significant concerns about algorithmic bias, which can perpetuate discrimination, inequity, and prejudice. This chapter draws on the Australian Robodebt harms, where automated systems incorrectly calculated welfare debt, causing severe physical and mental harm to many individuals to explore the implications of algorithmic bias within educational systems globally. It conceptually analyses how such biases can manifest in initial teacher education programs, leading to indirect forms of discrimination and impacting human rights. The chapter considers three critical encounters: (1) Automated Virtual Proctoring, (2) Talent Analytics for screening workplace behaviour and (3) Automated Mental Health and Wellness apps. Reflecting on these scenarios by aligning the harms caused by Robodebt, enables the author to propose future needs in initial teacher training to address and mitigate algorithmic bias and the risk of harm with emergent technologies such as generative Artificial Iintelligence (AI) or gen AI. It calls for greater discussion and debate on the commercialization of predictive analytics in education, aiming to empower educators worldwide to critically negotiate the integration of these technologies into their practices and policies.