Objectives To assess changes in peer-reviewed evidence on commercially available radiological artificial intelligence (AI) products from 2020 to 2023, as a follow-up to a 2020 review of 100 products. Materials and methods A literature review was conducted, covering January 2015 to March 2023, focusing on CE-certified radiological AI products listed on <a href="http://www.healthairegister.com">www.healthairegister.com</a>. Papers were categorised using the hierarchical model of efficacy: technical/diagnostic accuracy (levels 1–2), clinical decision-making and patient outcomes (levels 3–5), or socio-economic impact (level 6). Study features such as design, vendor independence, and multicentre/multinational data usage were also examined. Results By 2023, 173 CE-certified AI products from 90 vendors were identified, compared to 100 products in 2020. Products with peer-reviewed evidence increased from 36% to 66%, supported by 639 papers (up from 237). Diagnostic accuracy studies (level 2) remained predominant, though their share decreased from 65% to 57%. Studies addressing higher-efficacy levels (3–6) remained constant at 22% and 24%, with the number of products supported by such evidence increasing from 18% to 31%. Multicentre studies rose from 30% to 41% (p < 0.01). However, vendor-independent studies decreased (49% to 45%), as did multinational studies (15% to 11%) and prospective designs (19% to 16%), all with p > 0.05. Conclusion The increase in peer-reviewed evidence and higher levels of evidence per product indicate maturation in the radiological AI market. However, the continued focus on lower-efficacy studies and reductions in vendor independence, multinational data, and prospective designs highlight persistent challenges in establishing unbiased, real-world evidence. Key Points <Emphasis Type="BoldItalic">Question</Emphasis> Evaluating advancements in peer-reviewed evidence for CE-certified radiological AI products is crucial to understand their clinical adoption and impact. <Emphasis Type="BoldItalic">Findings</Emphasis> CE-certified AI products with peer-reviewed evidence increased from 36% in 2020 to 66% in 2023, but the proportion of higher-level evidence papers (~24%) remained unchanged. <Emphasis Type="BoldItalic">Clinical relevance</Emphasis> The study highlights increased validation of radiological AI products but underscores a continued lack of evidence on their clinical and socio-economic impact, which may limit these tools’ safe and effective implementation into clinical workflows. Graphical Abstract

Artificial intelligence in radiology: 173 commercially available products and their scientific evidence

期刊论文
出版模式：

开放获取
发表日期：

2025年7月24日

查看全文

Noa Antonissen,
Olga Tryfonos,
Ignas B. Houben,
Colin Jacobs,
Maarten de Rooij,
Kicky G. van Leeuwen

摘要

Objectives

To assess changes in peer-reviewed evidence on commercially available radiological artificial intelligence (AI) products from 2020 to 2023, as a follow-up to a 2020 review of 100 products.

Materials and methods

A literature review was conducted, covering January 2015 to March 2023, focusing on CE-certified radiological AI products listed on www.healthairegister.com. Papers were categorised using the hierarchical model of efficacy: technical/diagnostic accuracy (levels 1–2), clinical decision-making and patient outcomes (levels 3–5), or socio-economic impact (level 6). Study features such as design, vendor independence, and multicentre/multinational data usage were also examined.

Results

By 2023, 173 CE-certified AI products from 90 vendors were identified, compared to 100 products in 2020. Products with peer-reviewed evidence increased from 36% to 66%, supported by 639 papers (up from 237). Diagnostic accuracy studies (level 2) remained predominant, though their share decreased from 65% to 57%. Studies addressing higher-efficacy levels (3–6) remained constant at 22% and 24%, with the number of products supported by such evidence increasing from 18% to 31%. Multicentre studies rose from 30% to 41% (p < 0.01). However, vendor-independent studies decreased (49% to 45%), as did multinational studies (15% to 11%) and prospective designs (19% to 16%), all with p > 0.05.

Conclusion

The increase in peer-reviewed evidence and higher levels of evidence per product indicate maturation in the radiological AI market. However, the continued focus on lower-efficacy studies and reductions in vendor independence, multinational data, and prospective designs highlight persistent challenges in establishing unbiased, real-world evidence.

Key Points

Question Evaluating advancements in peer-reviewed evidence for CE-certified radiological AI products is crucial to understand their clinical adoption and impact.

Findings CE-certified AI products with peer-reviewed evidence increased from 36% in 2020 to 66% in 2023, but the proportion of higher-level evidence papers (~24%) remained unchanged.

Clinical relevance The study highlights increased validation of radiological AI products but underscores a continued lack of evidence on their clinical and socio-economic impact, which may limit these tools’ safe and effective implementation into clinical workflows.

Graphical Abstract