Collaborative Natural Language Processing for Automatic Scoring of Open-Ended Questions in Science Education
摘要
This chapter aims to explore the potential of Natural Language Processing (NLP) to assess the quality of answers to open-ended questions in science education effectively and objectively. These open-ended questions are designed to evaluate learners’ scientific causal reasoning. Given that assessing higher-order thinking skills, which help to engage students in meaningful learning, has become a focus in science education, automatic evaluation of constructed responses is increasingly emphasised. This is one of the reasons why more educators are using artificial intelligence to address the complex issues of automated assessments. Despite these strides, the automatic grading of open-ended questions remains a considerable challenge when the automatic grading is applied to different languages. This chapter tried to automatically grade open-ended questions by incorporating human educators and AI graders, in which AI graders provided valuable feedback to help educators refining input datasets, which in turn promoted optimal training for AI graders. The results suggested that AI graders were promising and achieved a high Receiver Operating Characteristic Curve (AUC-ROC) score of 0.949. This collaboration between human expertise and AI graders could enhance the accuracy of open-ended question assessments and improve efficiency in educational practices.