Explanations are essential to motivate AI-generated predictions in critical domains such as healthcare. To that end, interpretable surrogate models can provide post-hoc explanations for black-box machine learning (ML) models. When multiple black-box ML models (e.g., ensemble learning) are used, i.e., to overcome the prediction limitations of singular models, multiple surrogate models are similarly required, one per black-box model. Here, the problem lies in extracting a coherent explanation for a given prediction from multiple surrogates. An explanation can be seen as a “decision path” with sequentially ordered conditions. We can thus map this problem to process mining, mapping decision paths to traces, with the decision path and surrogate model as unique case, conditions as events, and their ordering as timestamps. We apply process mining to identify coherent explanations that represent the spectrum of decision logic encoded by different surrogate models. Our case study targets kidney transplant outcome predictions. We transform the surrogate models’ decision paths into an event log and analyze them using process discovery and trace clustering techniques. Our initial results reveal common and meaningful patterns among the decision paths. While prior research has focused on leveraging explainable AI (XAI) techniques to improve process mining, our paper takes the opposite approach, meaning employing process mining to enhance the interpretability of surrogate models. To the best of our knowledge, this is the first work to apply process discovery to surrogate decision paths.