Multistage adaptive testing (MST) designs are becoming common in international large-scale assessments (ILSAs). For example, an MST was implemented for the Programme for the International Assessment of Adult Competencies (PIAAC) in 2012 for about 40 countries and in the 2018 and 2022 cycles of the Programme for International Student Assessment (PISA) for more than 70 countries. Using the example of PISA 2018 in the domain of Reading, this chapter addresses some important psychometric considerations in designing and implementing MST and analyzing the collected data in the context of ILSAs. Specifically, this chapter presents 1) the conditions considered when introducing MST, 2) how the items and units (i.e., a set of items) were developed and assigned, and 3) the aspects that were monitored and examined, including the performance of the MST delivery platform, decisions for scaling procedures in analyzing the data, and alternative ways of determining the proportion correct under the MST design. The chapter concludes with a summary on utilizing and monitoring the MST in ILSAs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improved Test Designs and Multistage Adaptive Testing in Large-Scale Assessments

  • Kentaro Yamamoto,
  • Hyo Jeong Shin,
  • Frederic Robin,
  • Lale Khorramdel,
  • Laura Halderman

摘要

Multistage adaptive testing (MST) designs are becoming common in international large-scale assessments (ILSAs). For example, an MST was implemented for the Programme for the International Assessment of Adult Competencies (PIAAC) in 2012 for about 40 countries and in the 2018 and 2022 cycles of the Programme for International Student Assessment (PISA) for more than 70 countries. Using the example of PISA 2018 in the domain of Reading, this chapter addresses some important psychometric considerations in designing and implementing MST and analyzing the collected data in the context of ILSAs. Specifically, this chapter presents 1) the conditions considered when introducing MST, 2) how the items and units (i.e., a set of items) were developed and assigned, and 3) the aspects that were monitored and examined, including the performance of the MST delivery platform, decisions for scaling procedures in analyzing the data, and alternative ways of determining the proportion correct under the MST design. The chapter concludes with a summary on utilizing and monitoring the MST in ILSAs.