Towards More Reliable SQL Auto-grading: A Hybrid Approach Using LLMs, Intuitionistic Fuzzy Sets, and Traditional Methods
摘要
Automated SQL grading systems continue to face challenges in achieving high accuracy. While large language models (LLMs) offer new possibilities for semantic understanding, their probabilistic nature and occasional inconsistency limit their reliability for high-stakes, fully automated assessment, especially at scale. This paper proposes a hybrid grading framework that decomposes the SQL evaluation process into modular, interpretable stages–strategically combining traditional techniques, targeted LLM prompting, and a formal model of uncertainty. Rather than relying solely on LLMs for direct grading, we leverage their strengths in supporting subtasks such as test data generation and query equivalence suggestion–tasks where LLM errors are more easily identified and corrected. To adequately express partial correctness, we incorporate intuitionistic fuzzy sets (IFS) as a foundation, capturing distinct degrees of correctness and incorrectness. Our framework enables selective manual verification in cases with high uncertainty, while substantially reducing the overall human effort required. In contexts where full automation is not yet acceptable–such as high-stakes courses–the method provides a practical path toward scalable grading without compromising accuracy. This work aims not for marginal LLM accuracy gains, but for a robust, human-in-the-loop solution that balances automation with trustworthiness–paving the way for more scalable and interpretable educational technologies.