Scholarlink

Background <p>Licensing examinations play a crucial role in protecting the public by establishing a minimum level of competency for safe practice. Nevertheless, empirical evidence on the psychometric quality of the National Medical Licensing Examination (NMLE) is limited. This study aimed to evaluate the effectiveness of the NMLE in assessing medical graduates in Ethiopia.</p> Methods <p>This cross-sectional study assessed the psychometric validity of the NMLE using Classical Test Theory (CTT) and the two-parameter logistic (2PL) Item Response Theory (IRT) models. We assessed the difficulty of the questions, the exam’s ability to distinguish between different levels of ability, and its overall reliability. We evaluated model fit using Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and likelihood ratio tests. We conducted Differential Item Functioning (DIF) using Mantel-Haenszel, Lord’s test, Raju’s method, and logistic regression methods, and assessed item contributions based on factor loading and communality values. We used data from the February 2024 exam taken by 1,259 candidates across 19 schools.</p> Results <p>The analysis revealed that the exam demonstrated high overall reliability (α = 0.87) and correctly classified borderline candidates. However, several items didn’t effectively distinguish between candidates with different levels of performance (mean = 0.18), and a few questions showed potential bias against candidates from private medical colleges.</p> Conclusion <p>Our findings suggest that the examination was appropriate for assessing minimum competency, but it needs review to address potential bias. We recommend ongoing monitoring of item quality and enhancing faculty training to improve the quality of exam items.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Psychometric evaluation of the Ethiopian National Medical Licensing Examination: evidence from classical and item response theory analyses

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月9日

查看全文
  • Lalem Menber Belay,
  • Matthias Siebeck,
  • Martin R. Fischer,
  • Markus Berndt,
  • Tegbar Yigzaw Sendekie

摘要

Background

Licensing examinations play a crucial role in protecting the public by establishing a minimum level of competency for safe practice. Nevertheless, empirical evidence on the psychometric quality of the National Medical Licensing Examination (NMLE) is limited. This study aimed to evaluate the effectiveness of the NMLE in assessing medical graduates in Ethiopia.

Methods

This cross-sectional study assessed the psychometric validity of the NMLE using Classical Test Theory (CTT) and the two-parameter logistic (2PL) Item Response Theory (IRT) models. We assessed the difficulty of the questions, the exam’s ability to distinguish between different levels of ability, and its overall reliability. We evaluated model fit using Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and likelihood ratio tests. We conducted Differential Item Functioning (DIF) using Mantel-Haenszel, Lord’s test, Raju’s method, and logistic regression methods, and assessed item contributions based on factor loading and communality values. We used data from the February 2024 exam taken by 1,259 candidates across 19 schools.

Results

The analysis revealed that the exam demonstrated high overall reliability (α = 0.87) and correctly classified borderline candidates. However, several items didn’t effectively distinguish between candidates with different levels of performance (mean = 0.18), and a few questions showed potential bias against candidates from private medical colleges.

Conclusion

Our findings suggest that the examination was appropriate for assessing minimum competency, but it needs review to address potential bias. We recommend ongoing monitoring of item quality and enhancing faculty training to improve the quality of exam items.