Scholarlink

Background <p>The Fundamentals of Endoscopic Surgery (FES) manual skills exam is a simulation-based assessment originally developed on the GI Mentor II (GIM) with strong validity evidence. Surgical Science has since released the ENDO Mentor Suite (ES), for which validity evidence was previously demonstrated in a small, randomized study. However, there were minor scoring discrepancies for Loop Reduction and Mucosal Inspection. This follow-up study aims to determine if the ES introduces a threat to validity in the form of construct irrelevant variance through analysis of FES examinee performance.</p> Methods <p>Retrospective analysis of FES examinee data, including demographics and performance, was conducted following the integration of the ES as a testing platform in 2023. Task and total scores were obtained through a standardized computer-based algorithm. Examinees without self-reported demographics were excluded from analysis. Performance and pass rates were compared between ES and GIM examinees using standard statistical methods and sensitivity analyses within a non-inferiority framework.</p> Results <p>Of 1,121 FES examinees, 436 (39%) provided demographic information. Simulator groups were similar in demographics and endoscopy experience. There were small statistically discernable differences in Retroflexion and Tool Targeting scores between ES and GIM examinees. However, total scores (ES 70 ± 12 vs GIM 69 ± 11, <i>p</i> = 0.55) and pass rates (ES 88% vs GIM 83%, <i>p</i> = 0.25) were similar. Sensitivity analyses also confirmed a similar pattern of non-inferiority.</p> Conclusion <p>The ES does not introduce construct irrelevant variance, supporting its use in the FES manual skills exam. This is demonstrated by similar overall performance and pass rates between ES and GIM examinees. Additionally, previously reported variability in Loop Reduction and Mucosal Inspection were not redemonstrated within this larger sample. Given that over half of examinees were excluded from this analysis, measures to improve quality control on demographic reporting will be essential for additional validity investigations.</p> Graphical Abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Validity evidence for the new ENDO Mentor Suite and its use in the Fundamentals of Endoscopic Surgery examination

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月19日

查看全文
  • Noosha D. Deravi,
  • Payton M. Miller,
  • Maya L. Hunt,
  • Jessica Mischna,
  • Jose M. Martinez,
  • Eric M. Pauli,
  • E. Matthew Ritter

摘要

Background

The Fundamentals of Endoscopic Surgery (FES) manual skills exam is a simulation-based assessment originally developed on the GI Mentor II (GIM) with strong validity evidence. Surgical Science has since released the ENDO Mentor Suite (ES), for which validity evidence was previously demonstrated in a small, randomized study. However, there were minor scoring discrepancies for Loop Reduction and Mucosal Inspection. This follow-up study aims to determine if the ES introduces a threat to validity in the form of construct irrelevant variance through analysis of FES examinee performance.

Methods

Retrospective analysis of FES examinee data, including demographics and performance, was conducted following the integration of the ES as a testing platform in 2023. Task and total scores were obtained through a standardized computer-based algorithm. Examinees without self-reported demographics were excluded from analysis. Performance and pass rates were compared between ES and GIM examinees using standard statistical methods and sensitivity analyses within a non-inferiority framework.

Results

Of 1,121 FES examinees, 436 (39%) provided demographic information. Simulator groups were similar in demographics and endoscopy experience. There were small statistically discernable differences in Retroflexion and Tool Targeting scores between ES and GIM examinees. However, total scores (ES 70 ± 12 vs GIM 69 ± 11, p = 0.55) and pass rates (ES 88% vs GIM 83%, p = 0.25) were similar. Sensitivity analyses also confirmed a similar pattern of non-inferiority.

Conclusion

The ES does not introduce construct irrelevant variance, supporting its use in the FES manual skills exam. This is demonstrated by similar overall performance and pass rates between ES and GIM examinees. Additionally, previously reported variability in Loop Reduction and Mucosal Inspection were not redemonstrated within this larger sample. Given that over half of examinees were excluded from this analysis, measures to improve quality control on demographic reporting will be essential for additional validity investigations.

Graphical Abstract