<p>Framing questions about a program’s source code requires not only considerable subject matter expertise, but also sufficient time and effort. While a program source code can have multiple features such as the syntactic information through abstract syntax tree and the natural sequence of code itself, existing techniques rely only on code comments to generate questions. To automatically generate questions from program source code using multi-feature information retrieval approach. First, a CodeBERT encoder has been used to understand the source code representation. Subsequently a transformer-based decoder is used to generate questions from the learned representation. Multiple input features of code such as syntax and semantics, have been used for improved representation learning. Subsequently, a case study is conducted to evaluate the effectiveness of the generated questions in terms of its capability to differentiate students of different proficiency levels. The questions generated using the proposed approach demonstrate higher quality compared to those produced by existing methods. Moreover, a case study on real world programming assessment showed that the generated questions are effective in discriminating participants of higher proficiency level from the lower level. The results of this study have the potential to enhance the effectiveness of educational assessments. The effectiveness of the proposed approach is realized through a case study.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic Question Generation from Program Source Code for Educational Assessment

  • Jyoti Prakash Meher,
  • Rajib Mall

摘要

Framing questions about a program’s source code requires not only considerable subject matter expertise, but also sufficient time and effort. While a program source code can have multiple features such as the syntactic information through abstract syntax tree and the natural sequence of code itself, existing techniques rely only on code comments to generate questions. To automatically generate questions from program source code using multi-feature information retrieval approach. First, a CodeBERT encoder has been used to understand the source code representation. Subsequently a transformer-based decoder is used to generate questions from the learned representation. Multiple input features of code such as syntax and semantics, have been used for improved representation learning. Subsequently, a case study is conducted to evaluate the effectiveness of the generated questions in terms of its capability to differentiate students of different proficiency levels. The questions generated using the proposed approach demonstrate higher quality compared to those produced by existing methods. Moreover, a case study on real world programming assessment showed that the generated questions are effective in discriminating participants of higher proficiency level from the lower level. The results of this study have the potential to enhance the effectiveness of educational assessments. The effectiveness of the proposed approach is realized through a case study.