SmartEval: Advancing Automated Evaluation of Textual Answer Scripts with Transformer Models
摘要
Assessment and evaluation play a multifaceted role in the education sector. They not only measure students’ knowledge but also contribute to the overall quality of education. Manual evaluation of handwritten answer scripts tends to become a tedious and time-consuming process leading to delayed feedback. It is also an ineffective process because it is susceptible to human error and subjectivity. It may introduce bias if the graders unintentionally favor certain content or writing styles that don’t align with fair evaluation. Existing solutions to address these challenges have typically relied on keyword-based similarity, wherein evaluators are required to input specific keywords for comparison. However, this approach has limitations and may not capture the nuances of student responses effectively. In the context of the research we are presenting, our objective is to transcend the reliance on keywords as input and the evaluation of answers based solely on the presence or absence of such keywords. Instead, we propose the utilization of transformer models to assess similarity, offering a more sophisticated and robust methodology for evaluating student answers. In this paper, we comprehensively describe a web-based system for evaluating handwritten answers, developed as a result of a comparative study detailed herein.