In this paper we discuss a “last mile” problem applied to sports analytics. Specifically, ensuring that models with high aggregate performance are reliable in rare edge case scenarios, which are often those that matter most to analysts and fans. We center this discussion around Expected Goals (xG), the most widely known predictive statistic in soccer. We show how combining statistical and sports domain expertise can be used to develop smarter features to condense the feature space, apply monotonic constraints to impose rationality on the learnt relationship between important geometric features, and use data augmentation to increase the exposure of the model to rare shot types. We develop a set of low-level evaluation criteria which show how model performance improves on rare shot archetypes and reacts more realistically to perturbations of input data, particularly concerning the geometric relationship between strikers and the goalkeeper, an important and highly sensitive feature in the model.