Scholarlink

<p>Poor generalizability continues to hinder mapping brain activity to behavior using human neuroimaging data. A potential solution is predictive modeling, which evaluates generalizability in unseen data from the same dataset. However, predictive models often fail external validation—a stricter test of generalizability involving evaluation in an independent dataset. In this Perspective, we explain how evaluating generalizability via external validation can improve replicability (illusory generalizability and bias) and reproducibility (data leakage and data manipulations). We also provide advice on statistical power, dataset shift and training models. A model’s success in external validation provides evidence for its generalizability, but interpretations depend on the population characteristics of the training and external datasets. Even failed external validation constitutes an opportunity for scientific insight or methodological adjustments. Sharing data and models, combined with standalone external validation studies, will increase the prevalence of external validation. In turn, an increased focus on external validation can drive more generalizable, replicable and reproducible neuroimaging results.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

External validation improves generalizability, replicability and reproducibility in predictive models for neuroimaging

  • 期刊论文

  • 发表日期：

    2026年5月27日

查看全文
  • Matthew Rosenblatt,
  • Maya L. Foster,
  • Brendan D. Adkinson,
  • Link Tejavibulya,
  • Milana Khaitova,
  • Jean Ye,
  • Huili Sun,
  • Raimundo X. Rodriguez,
  • Chris C. Camp,
  • Ash Chinta,
  • Marie C. McCusker,
  • Ling Han,
  • Christopher T. Fields,
  • Saloni Mehta,
  • Dustin Scheinost

摘要

Poor generalizability continues to hinder mapping brain activity to behavior using human neuroimaging data. A potential solution is predictive modeling, which evaluates generalizability in unseen data from the same dataset. However, predictive models often fail external validation—a stricter test of generalizability involving evaluation in an independent dataset. In this Perspective, we explain how evaluating generalizability via external validation can improve replicability (illusory generalizability and bias) and reproducibility (data leakage and data manipulations). We also provide advice on statistical power, dataset shift and training models. A model’s success in external validation provides evidence for its generalizability, but interpretations depend on the population characteristics of the training and external datasets. Even failed external validation constitutes an opportunity for scientific insight or methodological adjustments. Sharing data and models, combined with standalone external validation studies, will increase the prevalence of external validation. In turn, an increased focus on external validation can drive more generalizable, replicable and reproducible neuroimaging results.