Scholarlink

<p>This study introduces a data-driven cut score method that integrates Item Response Theory (IRT), cluster analysis, and Gaussian-Otsu thresholding to objectively determine cut scores. The proposed approach, abbreviated as DDCS, provides a fully repeatable and non-judgmental framework that eliminates the need for expert panels used in traditional methods such as Angoff, while maintaining the conceptual goal of identifying a borderline competency threshold. In this study, the performance of DDCS was compared with the traditional Angoff approach, both using simulated data generated under different sample sizes, test lengths, and distributions of ability, difficulty, and discrimination parameters, and with realistic data. The two methods were evaluated using metrics such as accuracy, sensitivity, specificity, bias, absolute bias, and RMSE. Simulation results showed that the DDCS method outperformed the traditional Angoff approach in 88.9% of all scenarios, with particularly notable improvements in larger samples, longer tests, and asymmetric ability distributions, reducing RMSE by an average of 35–40%. Analyses of real test data further confirmed these findings: the DDCS method consistently achieved classification accuracy above 93%, whereas the traditional Angoff method ranged from 62.8% to 92.2%. Moreover, the DDCS framework maintained a balanced trade-off between sensitivity and specificity, while the traditional method demonstrated nearly perfect sensitivity but poor specificity (0.201–0.628). Overall, the DDCS framework offers a robust, transparent, and scientifically grounded alternative to expert-based standard-setting methods that by removing subjective judgment and employing an algorithmic, data-driven process, it enhances fairness, reduces cognitive burden, and ensures consistent and defensible cut score determination in high-stakes testing contexts.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Data-driven cut score method based on IRT, clustering, and gaussian otsu thresholding

  • 期刊论文

  • 发表日期：

    2026年3月9日

查看全文
  • Maryam Parsaeian,
  • Ebrahim Khodaie,
  • Habib Naderi

摘要

This study introduces a data-driven cut score method that integrates Item Response Theory (IRT), cluster analysis, and Gaussian-Otsu thresholding to objectively determine cut scores. The proposed approach, abbreviated as DDCS, provides a fully repeatable and non-judgmental framework that eliminates the need for expert panels used in traditional methods such as Angoff, while maintaining the conceptual goal of identifying a borderline competency threshold. In this study, the performance of DDCS was compared with the traditional Angoff approach, both using simulated data generated under different sample sizes, test lengths, and distributions of ability, difficulty, and discrimination parameters, and with realistic data. The two methods were evaluated using metrics such as accuracy, sensitivity, specificity, bias, absolute bias, and RMSE. Simulation results showed that the DDCS method outperformed the traditional Angoff approach in 88.9% of all scenarios, with particularly notable improvements in larger samples, longer tests, and asymmetric ability distributions, reducing RMSE by an average of 35–40%. Analyses of real test data further confirmed these findings: the DDCS method consistently achieved classification accuracy above 93%, whereas the traditional Angoff method ranged from 62.8% to 92.2%. Moreover, the DDCS framework maintained a balanced trade-off between sensitivity and specificity, while the traditional method demonstrated nearly perfect sensitivity but poor specificity (0.201–0.628). Overall, the DDCS framework offers a robust, transparent, and scientifically grounded alternative to expert-based standard-setting methods that by removing subjective judgment and employing an algorithmic, data-driven process, it enhances fairness, reduces cognitive burden, and ensures consistent and defensible cut score determination in high-stakes testing contexts.