Revisiting Automatic Essay Assessment: A Relative Approach
摘要
Automatic essay assessment has remained a challenging problem in natural language processing (NLP) for over fifty years. Traditionally framed as a regression task that predicts examiner-assigned scores, this work instead adopts a relative assessment perspective. The task is redefined as pairwise ranking, where essays are compared using standard assessment criteria, but their exact grades are assumed to be unknown and ultimately unimportant. Three core contributions are presented. First, an architecture is proposed for pairwise essay ranking using labeled quality criteria, with two fusion strategies explored. Second, an experimental setup for relative ranking is redefined based on a benchmark dataset originally intended for absolute scoring. Third, empirical comparisons are reported across architectural components and design choices. Results indicate that data-level fusion using LLaMA-based embeddings and a gradient-boosted classifier achieves the highest accuracy. A model-level fusion approach with a Siamese neural network and probabilistic loss also yields competitive performance, highlighting its potential for further research. This relative assessment framework offers practical value for student progress tracking and norm-referenced grading in educational contexts.