Automatic essay assessment has remained a challenging problem in natural language processing (NLP) for over fifty years. Traditionally framed as a regression task that predicts examiner-assigned scores, this work instead adopts a relative assessment perspective. The task is redefined as pairwise ranking, where essays are compared using standard assessment criteria, but their exact grades are assumed to be unknown and ultimately unimportant. Three core contributions are presented. First, an architecture is proposed for pairwise essay ranking using labeled quality criteria, with two fusion strategies explored. Second, an experimental setup for relative ranking is redefined based on a benchmark dataset originally intended for absolute scoring. Third, empirical comparisons are reported across architectural components and design choices. Results indicate that data-level fusion using LLaMA-based embeddings and a gradient-boosted classifier achieves the highest accuracy. A model-level fusion approach with a Siamese neural network and probabilistic loss also yields competitive performance, highlighting its potential for further research. This relative assessment framework offers practical value for student progress tracking and norm-referenced grading in educational contexts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Revisiting Automatic Essay Assessment: A Relative Approach

  • Anda Leşeanu,
  • İbrahim Rıza Hallaç,
  • Burçin Buket Oğul,
  • Hasan Oğul

摘要

Automatic essay assessment has remained a challenging problem in natural language processing (NLP) for over fifty years. Traditionally framed as a regression task that predicts examiner-assigned scores, this work instead adopts a relative assessment perspective. The task is redefined as pairwise ranking, where essays are compared using standard assessment criteria, but their exact grades are assumed to be unknown and ultimately unimportant. Three core contributions are presented. First, an architecture is proposed for pairwise essay ranking using labeled quality criteria, with two fusion strategies explored. Second, an experimental setup for relative ranking is redefined based on a benchmark dataset originally intended for absolute scoring. Third, empirical comparisons are reported across architectural components and design choices. Results indicate that data-level fusion using LLaMA-based embeddings and a gradient-boosted classifier achieves the highest accuracy. A model-level fusion approach with a Siamese neural network and probabilistic loss also yields competitive performance, highlighting its potential for further research. This relative assessment framework offers practical value for student progress tracking and norm-referenced grading in educational contexts.