Scholarlink

Background <p>The diagnosis of headache disorders remains a clinical challenge, particularly for non-specialists, due to the complexity of the International Classification of Headache Disorders, 3rd edition (ICHD-3), and the absence of biomarkers. Large language models (LLMs) represent a promising tool to support accurate and scalable diagnostic classification, especially in resource-limited settings.</p> Objective <p>To validate the performance of a free, multilingual clinical decision support platform—Head.AI—designed to classify headache cases using GPT-4o and a structured implementation of ICHD-3.</p> Methods <p>We conducted an independent validation using 315 expert-generated vignettes representing 215 ICHD-3 diagnoses, input into Head.AI and three other platforms (Claude Sonnet 4.0, Grok 3.0, and Gemini 2.5). Outcomes included diagnostic accuracy (rank of correct diagnosis), calibration, and citation rate.</p> Results <p>The algorithm correctly identified the top diagnosis in 89.5% of cases (vs. 74–80% in comparators), with a citation rate &gt;97% and calibration (Brier score 0.153). It maintained consistent performance across primary and secondary headaches and achieved first-hypothesis accuracy &gt;74% in difficult cases. Logistic regression confirmed Head.AI had significantly higher odds of correct classification (ORs vs. comparators: 2.04–2.86; all p &lt; 0.01).</p> Conclusion <p>Our algorithm demonstrated high diagnostic accuracy across a broad spectrum of headache disorders, exceeding the performance reported in prior studies, though direct comparison should be interpreted with caution due to methodological differences. Its public availability, structured knowledge base, and educational potential make it a valuable contribution to AI-assisted headache care. The platform is freely accessible at <a href="https://www.head-ai.com.br">www.head-ai.com.br</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Validation of an AI-Based platform for structured diagnosis of headache disorders using ICHD-3 criteria

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月27日

查看全文
  • João Brainer Clares de Andrade,
  • Thiago Bulhões da Silva Costa,
  • Júlia Lima Vasconcelos,
  • Thiago Luís Marques Lopes,
  • Mateus Dutra Balsells,
  • Vinícius Luiz Cristofolini,
  • Sophia Oliveira Querobin,
  • Flavio Moura Rezende Filho

摘要

Background

The diagnosis of headache disorders remains a clinical challenge, particularly for non-specialists, due to the complexity of the International Classification of Headache Disorders, 3rd edition (ICHD-3), and the absence of biomarkers. Large language models (LLMs) represent a promising tool to support accurate and scalable diagnostic classification, especially in resource-limited settings.

Objective

To validate the performance of a free, multilingual clinical decision support platform—Head.AI—designed to classify headache cases using GPT-4o and a structured implementation of ICHD-3.

Methods

We conducted an independent validation using 315 expert-generated vignettes representing 215 ICHD-3 diagnoses, input into Head.AI and three other platforms (Claude Sonnet 4.0, Grok 3.0, and Gemini 2.5). Outcomes included diagnostic accuracy (rank of correct diagnosis), calibration, and citation rate.

Results

The algorithm correctly identified the top diagnosis in 89.5% of cases (vs. 74–80% in comparators), with a citation rate >97% and calibration (Brier score 0.153). It maintained consistent performance across primary and secondary headaches and achieved first-hypothesis accuracy >74% in difficult cases. Logistic regression confirmed Head.AI had significantly higher odds of correct classification (ORs vs. comparators: 2.04–2.86; all p < 0.01).

Conclusion

Our algorithm demonstrated high diagnostic accuracy across a broad spectrum of headache disorders, exceeding the performance reported in prior studies, though direct comparison should be interpreted with caution due to methodological differences. Its public availability, structured knowledge base, and educational potential make it a valuable contribution to AI-assisted headache care. The platform is freely accessible at www.head-ai.com.br.