Scholarlink

Background <p>Headache disorders are frequently misdiagnosed. We aimed to systematically evaluate the diagnostic accuracy, methodological quality, and clinical applicability of artificial intelligence (AI) and machine learning (ML) models for classifying adult headache disorders against clinician diagnoses using the International Classification of Headache Disorders (ICHD) criteria.</p> Methods <p>In this systematic review, we searched PubMed, Embase, and the Cochrane Library (January 2015–December 2025) for AI/ML diagnostic headache studies. Two reviewers extracted data and assessed risk of bias using the QUADAS-2 tool with the QUADAS-AI extension. Main outcomes were sensitivity, specificity, area under the receiver operating characteristic curve (AUC-ROC), and risk of bias.</p> Results <p>We included 74 studies encompassing 154,856 participants. Models utilized traditional ML (<i>n</i> = 47), deep learning (<i>n</i> = 18), and hybrid or rule-based approaches (<i>n</i> = 9). Data inputs included neuroimaging (<i>n</i> = 27), multimodal datasets (<i>n</i> = 20), neurophysiological signals (<i>n</i> = 17), and clinical questionnaires (<i>n</i> = 10). Only 4 studies performed independent external validation. Overall sensitivity ranged from 47.5% to 100.0%, specificity from 50.4% to 100.0%, and AUC-ROC from 0.658 to 1.000. Models using structured questionnaires reported realistic accuracies (74–86%), whereas neuroimaging models frequently produced near-perfect, likely overfit estimates. Most studies (65 of 74) exhibited a high risk of bias driven by artificial case–control designs, data leakage, and absent external validation.</p> Conclusions <p>AI-based headache diagnostic systems demonstrate promising but highly variable accuracy. Pervasive methodological flaws—specifically severe spectrum bias, data leakage, and a lack of independent external validation—currently preclude clinical implementation. Future studies require prospective, real-world validation to safely integrate these tools into practice.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

‘Man vs. Machine: can ML algorithms diagnose headaches as accurately as clinicians? A systematic review.’

  • 期刊论文

  • 发表日期：

    2026年5月26日

查看全文
  • Appukutty Manickam,
  • Abirami Valliappan

摘要

Background

Headache disorders are frequently misdiagnosed. We aimed to systematically evaluate the diagnostic accuracy, methodological quality, and clinical applicability of artificial intelligence (AI) and machine learning (ML) models for classifying adult headache disorders against clinician diagnoses using the International Classification of Headache Disorders (ICHD) criteria.

Methods

In this systematic review, we searched PubMed, Embase, and the Cochrane Library (January 2015–December 2025) for AI/ML diagnostic headache studies. Two reviewers extracted data and assessed risk of bias using the QUADAS-2 tool with the QUADAS-AI extension. Main outcomes were sensitivity, specificity, area under the receiver operating characteristic curve (AUC-ROC), and risk of bias.

Results

We included 74 studies encompassing 154,856 participants. Models utilized traditional ML (n = 47), deep learning (n = 18), and hybrid or rule-based approaches (n = 9). Data inputs included neuroimaging (n = 27), multimodal datasets (n = 20), neurophysiological signals (n = 17), and clinical questionnaires (n = 10). Only 4 studies performed independent external validation. Overall sensitivity ranged from 47.5% to 100.0%, specificity from 50.4% to 100.0%, and AUC-ROC from 0.658 to 1.000. Models using structured questionnaires reported realistic accuracies (74–86%), whereas neuroimaging models frequently produced near-perfect, likely overfit estimates. Most studies (65 of 74) exhibited a high risk of bias driven by artificial case–control designs, data leakage, and absent external validation.

Conclusions

AI-based headache diagnostic systems demonstrate promising but highly variable accuracy. Pervasive methodological flaws—specifically severe spectrum bias, data leakage, and a lack of independent external validation—currently preclude clinical implementation. Future studies require prospective, real-world validation to safely integrate these tools into practice.