Enhancing Credit Risk Prediction Using Stacked Ensemble and Quantum Approaches on PySpark
摘要
Objective: Accurate prediction of credit risk is essential for financial organizations to handle defaults efficiently. As data complexity increases, the integration of traditional and novel AI models becomes imperative. This study improves the prediction of credit defaults with a hybrid quantum-classical stacking methodology. Methods: This research utilizes an interdisciplinary approach that integrates quantum machine learning with conventional ensemble methodologies with the Pyspark framework. The classical layer consists of Light Gradient Boosting Machine (LGBM), Extreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost), and HistGradientBoosting (HGB) classifiers optimized using Optuna. An ensemble is augmented with a variational quantum classifier developed using PennyLane. Stacking is executed via LGBM as a meta-learner. Principal Component Analysis (PCA) is used for dimensionality reduction, while SMOTETomek tackles class imbalance. Results: In the credit data set, the mixed-stacked model attained an accuracy 94.47%, with a precision 93.13%, a recall 97.60%, F1-score 95.31% and an AUC 98.33%. The incorporation of quantum characteristics improved the variety and resilience of the model, resulting in more refined decision limits. Conclusion: The findings indicate that including quantum classifiers in standard ensemble frameworks can significantly enhance the effectiveness of credit risk modelling, particularly in unbalanced and high-dimensional financial datasets. Significance: This adapted methodology presents an innovative direction in credit scoring by integrating classical elucidative models with quantum AI. Improve predictive accuracy and model interpretability and facilitate the development of advanced financial risk assessment tools.