This chapter aims to explore the potential of Natural Language Processing (NLP) to assess the quality of answers to open-ended questions in science education effectively and objectively. These open-ended questions are designed to evaluate learners’ scientific causal reasoning. Given that assessing higher-order thinking skills, which help to engage students in meaningful learning, has become a focus in science education, automatic evaluation of constructed responses is increasingly emphasised. This is one of the reasons why more educators are using artificial intelligence to address the complex issues of automated assessments. Despite these strides, the automatic grading of open-ended questions remains a considerable challenge when the automatic grading is applied to different languages. This chapter tried to automatically grade open-ended questions by incorporating human educators and AI graders, in which AI graders provided valuable feedback to help educators refining input datasets, which in turn promoted optimal training for AI graders. The results suggested that AI graders were promising and achieved a high Receiver Operating Characteristic Curve (AUC-ROC) score of 0.949. This collaboration between human expertise and AI graders could enhance the accuracy of open-ended question assessments and improve efficiency in educational practices.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Collaborative Natural Language Processing for Automatic Scoring of Open-Ended Questions in Science Education

  • Van T. Hoang Ngo,
  • John J. H. Lin,
  • Chun-Yen Chang

摘要

This chapter aims to explore the potential of Natural Language Processing (NLP) to assess the quality of answers to open-ended questions in science education effectively and objectively. These open-ended questions are designed to evaluate learners’ scientific causal reasoning. Given that assessing higher-order thinking skills, which help to engage students in meaningful learning, has become a focus in science education, automatic evaluation of constructed responses is increasingly emphasised. This is one of the reasons why more educators are using artificial intelligence to address the complex issues of automated assessments. Despite these strides, the automatic grading of open-ended questions remains a considerable challenge when the automatic grading is applied to different languages. This chapter tried to automatically grade open-ended questions by incorporating human educators and AI graders, in which AI graders provided valuable feedback to help educators refining input datasets, which in turn promoted optimal training for AI graders. The results suggested that AI graders were promising and achieved a high Receiver Operating Characteristic Curve (AUC-ROC) score of 0.949. This collaboration between human expertise and AI graders could enhance the accuracy of open-ended question assessments and improve efficiency in educational practices.