Improved Test Designs and Multistage Adaptive Testing in Large-Scale Assessments
摘要
Multistage adaptive testing (MST) designs are becoming common in international large-scale assessments (ILSAs). For example, an MST was implemented for the Programme for the International Assessment of Adult Competencies (PIAAC) in 2012 for about 40 countries and in the 2018 and 2022 cycles of the Programme for International Student Assessment (PISA) for more than 70 countries. Using the example of PISA 2018 in the domain of Reading, this chapter addresses some important psychometric considerations in designing and implementing MST and analyzing the collected data in the context of ILSAs. Specifically, this chapter presents 1) the conditions considered when introducing MST, 2) how the items and units (i.e., a set of items) were developed and assigned, and 3) the aspects that were monitored and examined, including the performance of the MST delivery platform, decisions for scaling procedures in analyzing the data, and alternative ways of determining the proportion correct under the MST design. The chapter concludes with a summary on utilizing and monitoring the MST in ILSAs.