This chapter outlines the methodology and results of the pretesting phase conducted to refine the survey instruments used for evaluating the integration of generative AI (GenAI) in the master’s courses M562 and M563. Pretesting was a crucial step to ensure that the immediate and follow-up evaluation questionnaires met high standards across several dimensions: clarity and comprehension of questions, appropriateness and acceptability of response options, impartiality and lack of bias, overall quality of the questionnaire, clarity and sufficiency of instructions, logical flow of questions, and time required to complete the survey. By involving a diverse group of master’s students and using both quantitative feedback and follow-up interviews, the process identified ambiguities, cultural nuances, and practical improvements to strengthen the instruments. The refinements and insights presented in this chapter aim to support peers who wish to reuse or adapt the pretesting approach and survey instruments for similar educational interventions involving GenAI or related technologies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Pretesting of Survey Instruments: Methodology and Results

  • Varun Gupta,
  • Chetna Gupta

摘要

This chapter outlines the methodology and results of the pretesting phase conducted to refine the survey instruments used for evaluating the integration of generative AI (GenAI) in the master’s courses M562 and M563. Pretesting was a crucial step to ensure that the immediate and follow-up evaluation questionnaires met high standards across several dimensions: clarity and comprehension of questions, appropriateness and acceptability of response options, impartiality and lack of bias, overall quality of the questionnaire, clarity and sufficiency of instructions, logical flow of questions, and time required to complete the survey. By involving a diverse group of master’s students and using both quantitative feedback and follow-up interviews, the process identified ambiguities, cultural nuances, and practical improvements to strengthen the instruments. The refinements and insights presented in this chapter aim to support peers who wish to reuse or adapt the pretesting approach and survey instruments for similar educational interventions involving GenAI or related technologies.