Scholarlink

Code assessment in computer science education is a time-consuming process requiring evaluation of both functionality and coding style. To overcome this problems, we present a novel approach that utilizes Large Language Models (LLMs) for rubric-based code evaluation, addressing the limitations of current automated tools that rely primarily on test cases and code similarity metrics, which sometimes fail to capture more nuanced aspects of student submissions. Our method enables LLMs to analyze student code against instructor-defined rubrics, generating inline, contextualized feedback without altering the original code. This may allow instructors to efficiently review, refine, and finalize grades while maintaining pedagogical oversight. We evaluated the grading performance of five state-of-the-art LLMs (Claude-3.5 Sonnet, GPT-4o, Grok-2, LLaMA-3.3 70B, and DeepSeek-V3) against human graders using a dataset of over 500 coding exercises from an introductory university programming course. Our findings show that Claude-3.5 Sonnet exhibits the highest alignment with human grading, consistently outperforming other models across different exercise types. The dataset and code for reproducibility are available at: https://doi.org/10.5281/zenodo.14879148 .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leveraging Large Language Models to Assist Teachers in Code Grading

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Edoardo Cipriano,
  • Alessio Ferrato,
  • Carla Limongelli,
  • Daniele Schicchi,
  • Davide Taibi

摘要

Code assessment in computer science education is a time-consuming process requiring evaluation of both functionality and coding style. To overcome this problems, we present a novel approach that utilizes Large Language Models (LLMs) for rubric-based code evaluation, addressing the limitations of current automated tools that rely primarily on test cases and code similarity metrics, which sometimes fail to capture more nuanced aspects of student submissions. Our method enables LLMs to analyze student code against instructor-defined rubrics, generating inline, contextualized feedback without altering the original code. This may allow instructors to efficiently review, refine, and finalize grades while maintaining pedagogical oversight. We evaluated the grading performance of five state-of-the-art LLMs (Claude-3.5 Sonnet, GPT-4o, Grok-2, LLaMA-3.3 70B, and DeepSeek-V3) against human graders using a dataset of over 500 coding exercises from an introductory university programming course. Our findings show that Claude-3.5 Sonnet exhibits the highest alignment with human grading, consistently outperforming other models across different exercise types. The dataset and code for reproducibility are available at: https://doi.org/10.5281/zenodo.14879148 .