Generative artificial intelligence plays an important role in our recent advances in many fields including education. As generative models, large language models such as BERT, T5, and GPT have contributed much to automatic distractor generation for better preparing multiple-choice questions for educational assessment. However, different hyperparameter settings may lead to an avoidable trade-off between the performance of the distractor generation models and the diversity of their resulting distractors, which has not yet been thoroughly examined. Furthermore, the effectiveness of the distractor generation models might be affected by their decoding strategies. To address these issues, we propose a new solution based on T5-large-disjoint with diverse beam search to automatically generate diverse distractors for multiple-choice questions. Via the experimental results on our processed RACE dataset, our solution outperforms the existing ones with higher BLEU, ROUGE-L, and METEOR values, showing effective distractor generation. On the other hand, our work is the first study that confirms diverse beam search as a more appropriate decoding strategy for diverse distractor generation compared to other decoding ones. Above all, our solution has created more accurate distractors than the others. As a result, diverse distraction generation has been resolved effectively.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Effective Large Language Model-Based Diverse Distractor Generation for Multiple-Choice Questions

  • Duy Nguyen,
  • Chau Vo,
  • Phung Nguyen

摘要

Generative artificial intelligence plays an important role in our recent advances in many fields including education. As generative models, large language models such as BERT, T5, and GPT have contributed much to automatic distractor generation for better preparing multiple-choice questions for educational assessment. However, different hyperparameter settings may lead to an avoidable trade-off between the performance of the distractor generation models and the diversity of their resulting distractors, which has not yet been thoroughly examined. Furthermore, the effectiveness of the distractor generation models might be affected by their decoding strategies. To address these issues, we propose a new solution based on T5-large-disjoint with diverse beam search to automatically generate diverse distractors for multiple-choice questions. Via the experimental results on our processed RACE dataset, our solution outperforms the existing ones with higher BLEU, ROUGE-L, and METEOR values, showing effective distractor generation. On the other hand, our work is the first study that confirms diverse beam search as a more appropriate decoding strategy for diverse distractor generation compared to other decoding ones. Above all, our solution has created more accurate distractors than the others. As a result, diverse distraction generation has been resolved effectively.