This contribution builds upon recent work by the author on investigating the ways in which epistemic defeat arises in machine learning (ML) and the ethical problems that it raises. The contribution presents an epistemological approach to the problems of opacity and algorithmic bias in machine learning by discussing them in terms of the forms of epistemic defeat that arise in them. A taxonomy of epistemic defeaters, including transparent, opaque, and inherited defeaters, is developed and employed for this purpose. The Black-Box problem is discussed, and a distinction between two construals of the ‘why’ question that defines the Black-Box problem is proposed. These construals are based upon a distinction between a request for an explanation of a model’s output in terms of a causal mechanism and an explanation in terms of reasons or justification. The latter is shown to be the more challenging problem that remains difficult to solve even if a solution to the former is available. Particular attention will be given to the phenomenon of inherited epistemic defeat, which is a form of epistemic defeat involved in algorithmic bias that is derived from biases encoded in training data. Such defeaters are inherited via training data and inculcated in machine learning models, thereby exacerbating and obfuscating the effect of their epistemic defeat. This contribution builds upon a recent collaborative project conducted on machine learning in midwifery by the author (Begley et al. 2021). Examples from that study are considered along with analogous examples from predictive policing. Recommendations are made in particular regarding how inherited epistemic defeat should best be addressed.