Optimized Stacking Ensemble Framework with Feature Selection for Cardiovascular Disease Prediction
摘要
This study aims to enhance the accuracy and reliability of cardiovascular disease (CVD) prediction by integrating advanced feature selection techniques within a stacking ensemble framework. By utilizing clinical and demographic data from multiple sources, including the Hungarian, Switzerland, Statlog, Cleveland, and Long Beach VA datasets, the proposed method focuses on identifying the most relevant features to reduce noise and computational complexity while leveraging the strengths of diverse predictive models. The methodology involves constructing a stacking ensemble of various machine learning algorithms, including Extra Trees, Support Vector Machine, K-Nearest Neighbors, Decision Tree, Random Forest, Multi-Layer Perceptron, and Gradient Boosting, each trained using varied hyperparameters to capture different data patterns. To refine feature relevance and minimize overfitting three feature selection methods, Permutation Feature Importance (PFI), Pearson’s correlation coefficient (PCC), and Mutual Information-based Constructive Criterion (MICC) were applied, resulting in the identification of nine key attributes (cp, slope, fbs, chol, sex, age, exang, oldpeak, and mhr). These features yielded superior predictive performance compared to the top five features used in isolation, which led to reduced accuracy. The optimal model configuration used Random Forest, Extra Trees, MLP, and Gradient Boosting as the base classifiers, with Logistic Regression as the final classifier for the nine key attributes and achieved a mean ROC score of 0.92546 and an accuracy of 0.82563, validating the robustness of the model. Even when tested with reduced feature sets and different classifier combinations, the ensemble approach maintained strong performance. Additionally, to address data incompleteness, four datasets with varying levels of missingness were created, imputed, and evaluated for predictive consistency. Overall, this integrated strategy significantly improves CVD prediction by enhancing generalization, maintaining high accuracy, and reducing computational load, demonstrating its potential as a reliable tool for clinical decision support.