Predictive Maintenance (PdM) leverages data-driven models to anticipate equipment failures and optimize industrial operations. Explainability becomes a critical requirement for experts, as it helps identify the main issues leading to failures. This work proposes an explainable artificial intelligence (XAI) method suitable for online multi-label learning (MLL) in PdM. The approach is model-agnostic, although we base it on a recent and high-performing method by extending the MLHAT model to the PdM domain. To enhance user acceptance, a clustering method is incorporated to group misclassifications within each combined label, allowing the identification of representative instances to support the explanation of the model’s behavior at those moments. The incremental version of SAGE provides the method for online model inspection, evaluating how feature contributions vary around selected predictions. The proposal is tested on one public, multi-label PdM problem. Our experiment shows that the approach is able to extract representative insights into failure patterns in an online setting, identify the most challenging fault combinations to detect, and support understanding of the main factors contributing to prediction errors.