This paper explores using large language models (LLMs) like T5, BART, and GPT-4 Turbo to automatically generate feedback on primary and secondary school students’ essays. We constructed a dataset which consists of over 740 student essays and tutor feedback across different year levels, based on which we evaluated the performance of prevalent LLMs in feedback generation. After aligning automated evaluation metrics with educational standards in helpfulness, readability, acceptance, relevance, and specificity, we conducted further user studies to assess GPT-4’s effectiveness in personalised feedback generation. Our findings show that GPT-4 Turbo, especially when using well-designed prompts and reasoning strategies, outperforms models like T5 and BART in providing more readable feedback. Human evaluation also supports the readability and relevance of GPT-4 Turbo’s feedback, but lacks helpfulness and specificity compared to real tutor feedback.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generating Feedback for School Students Essay with Large Language Models

  • Dan Zhang,
  • Thuong Hoang,
  • Ye Zhu,
  • Rui Wang,
  • Paula Crouch

摘要

This paper explores using large language models (LLMs) like T5, BART, and GPT-4 Turbo to automatically generate feedback on primary and secondary school students’ essays. We constructed a dataset which consists of over 740 student essays and tutor feedback across different year levels, based on which we evaluated the performance of prevalent LLMs in feedback generation. After aligning automated evaluation metrics with educational standards in helpfulness, readability, acceptance, relevance, and specificity, we conducted further user studies to assess GPT-4’s effectiveness in personalised feedback generation. Our findings show that GPT-4 Turbo, especially when using well-designed prompts and reasoning strategies, outperforms models like T5 and BART in providing more readable feedback. Human evaluation also supports the readability and relevance of GPT-4 Turbo’s feedback, but lacks helpfulness and specificity compared to real tutor feedback.