We would like to have a wide range of explanations for the behaviour of machine learning systems. However, how should we understand these explanations? Typically, attempts to clarify what an explanations for questions such as ‘why am I getting this output for these inputs?’ have been approached from the philosophy of science, through an analogy with scientific (and often causal) explanations. I show that ML systems are best thought of as noncausal, specifically mathematical objects. We should therefore interpret these explanations differently, through analogy with mathematical explanations. I show that this still allows us to use much of the same theoretical apparatus, and argue that the asymmetry of many of the standard ML explanations can be accounted for in virtue of the link these systems have with concrete implementations.