Scholarlink

<p>In primary prevention of cardiovascular disease, risk prediction models require not only adequate discrimination but also robust stability and reliable probability calibration to support clinical decision-making. Using data from 4434 participants in the Framingham baseline cohort, this study systematically compared the performance of three representative prediction models for cardiovascular risk assessment: an L2-regularized logistic regression model, a lightly tuned XGBoost model, and an AutoML-XGBoost model with automated hyperparameter optimization. Model performance was evaluated using repeated stratified data splitting, with discrimination assessed by the area under the receiver operating characteristic curve (AUC), calibration evaluated using the Brier score and calibration curves, stability examined across repeated splits, and consistency of risk ranking analyzed between models. The results showed that the three models achieved comparable levels of discrimination, exhibiting similar AUC distributions across repeated validations, with no clear advantage of more complex models over the traditional regression approach. The logistic regression model demonstrated the smallest performance variability across data splits and achieved the lowest Brier score, indicating the most stable and well-calibrated probability estimates. The AutoML-XGBoost model outperformed the lightly tuned XGBoost model in terms of discrimination and stability, but its calibration performance remained slightly inferior to that of logistic regression. Risk ranking consistency analysis further revealed a high Spearman rank correlation of approximately 0.95 between logistic regression and AutoML-XGBoost, with an overlap of about 71% in individuals classified within the highest-risk decile, whereas the correlation between logistic regression and the lightly tuned XGBoost model was approximately 0.83, with only about 50% overlap in the highest-risk group. These findings indicate that increasing model complexity does not necessarily lead to improved overall predictive performance in classical cardiovascular risk prediction tasks, and that multidimensional performance evaluation is essential for the appropriate selection and clinical application of prediction models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Discrimination stability and calibration of cardiovascular risk prediction models in the Framingham baseline cohort

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月26日

查看全文
  • Junbo Zhang,
  • Tao Li

摘要

In primary prevention of cardiovascular disease, risk prediction models require not only adequate discrimination but also robust stability and reliable probability calibration to support clinical decision-making. Using data from 4434 participants in the Framingham baseline cohort, this study systematically compared the performance of three representative prediction models for cardiovascular risk assessment: an L2-regularized logistic regression model, a lightly tuned XGBoost model, and an AutoML-XGBoost model with automated hyperparameter optimization. Model performance was evaluated using repeated stratified data splitting, with discrimination assessed by the area under the receiver operating characteristic curve (AUC), calibration evaluated using the Brier score and calibration curves, stability examined across repeated splits, and consistency of risk ranking analyzed between models. The results showed that the three models achieved comparable levels of discrimination, exhibiting similar AUC distributions across repeated validations, with no clear advantage of more complex models over the traditional regression approach. The logistic regression model demonstrated the smallest performance variability across data splits and achieved the lowest Brier score, indicating the most stable and well-calibrated probability estimates. The AutoML-XGBoost model outperformed the lightly tuned XGBoost model in terms of discrimination and stability, but its calibration performance remained slightly inferior to that of logistic regression. Risk ranking consistency analysis further revealed a high Spearman rank correlation of approximately 0.95 between logistic regression and AutoML-XGBoost, with an overlap of about 71% in individuals classified within the highest-risk decile, whereas the correlation between logistic regression and the lightly tuned XGBoost model was approximately 0.83, with only about 50% overlap in the highest-risk group. These findings indicate that increasing model complexity does not necessarily lead to improved overall predictive performance in classical cardiovascular risk prediction tasks, and that multidimensional performance evaluation is essential for the appropriate selection and clinical application of prediction models.