Towards an Intelligent Automated Evaluation System for Python Powered by Large Language Models
摘要
Evaluation allows learners to confirm their understanding of Python and strengthens their learning by confronting them with practical scenarios. Regular assessments enable students to identify their strengths and weaknesses, thus facilitating targeted progress. Moreover, evaluation provides valuable feedback, guiding learners towards better practices and increased mastery of the language. However, manual assessment systems have certain constraints, such as the lengthy time required for design and correction, the risk of promoting cheating through identical exercises for all learners, and a potential lack of objectivity and consistency, especially when multiple graders are involved. This underscores the critical need for automated Python evaluation systems, which can instantly and objectively analyze and correct the code submitted by learners. These systems also excel in providing personalized feedback, which is crucial in guiding learners toward better practices and increased mastery of the language. In this article, we present the evolution of our Semi Code Writing Intelligent Tutoring System into an automated evaluation system based on language models, notably ChatGPT-3.5 Turbo. This system provides formative and summative assessments with varying levels of difficulty, including Parsons Problems, writing code from scratch, and semi-code writing, where learners must complete a partially provided code.