Bias–variance tradeoff decomposition-based machine learning model selection: application to credit risk analysis
摘要
Model selection is a critical task in machine learning (ML), as it entails identifying a model that generalizes effectively to unseen data. However, existing model selection techniques often suffer from high computational cost, limited interpretability, and sensitivity to hyperparameters. In this study, we propose a novel model selection framework grounded in the formal decomposition of the bias–variance tradeoff. The framework is structured along three critical dimensions: model complexity, training set size, and loss level. The proposed approach first determines the optimal training set size by detecting the point of convergence in the learning process, beyond which the loss stabilizes. Subsequently, the optimal model complexity is selected by quantifying and minimizing the bias–variance tradeoff. This approach was empirically validated using an XGBoost classifier in the context of credit risk classification. Experimental results demonstrate that the learning curve stabilizes at a specific training size, and that increasing model complexity beyond this point does not significantly enhance predictive performance. Furthermore, the selected model exhibited the lowest correlation and covariance between bias and variance, and outperformed alternative models during the test phase. These findings underscore the efficacy of the proposed framework in guiding principled model selection.