Scholarlink

<p>Ensuring that examination questions are pedagogically sound and aligned with the curriculum remains a key challenge in automated assessment. Most existing approaches focus on answer correctness or surface-level textual similarity, overlooking whether questions genuinely reflect syllabus content, require appropriate reasoning, or maintain structural coherence. This study presents SyllabiQA-RSNet, a syllabus-driven framework for automatically evaluating exam question quality. The model combines DeBERTa—large contextual embeddings with graph attention-based syllabus reasoning, capturing dependencies across syllabus sections and multi-step logical connections between questions and course content. Robustness scoring is formulated as a multi-task problem, with the model simultaneously predicting answers, generating reasoning chains, and producing continuous robustness scores. Evaluation on the SyllabusQA benchmark, comprising 5078 question–answer pairs from 63 university syllabi across STEM and non-STEM fields, demonstrates that SyllabiQA-RSNet substantially outperforms baseline models. It achieved a Pearson correlation of 0.79 and Spearman correlation of 0.76 with reference robustness scores, compared to 0.64 for DeBERTa-only and 0.58 for RoBERTa, while reducing RMSE to 0.082. The framework also enhanced answer prediction, particularly for reasoning-intensive questions, improving Exact Match scores by up to 7.2 points and producing higher-quality reasoning chains (+ 8.3 BLEU-4, + 8.8 ROUGE-L). Ablation studies confirmed the necessity of both syllabus graph reasoning and explicit robustness supervision, with their removal causing up to an 11-point drop in robustness correlation. These results indicate that explicitly modeling syllabus structure is critical for reliable, interpretable assessment of question quality, offering insights beyond simple answer correctness and enabling automated evaluation that aligns closely with pedagogical objectives.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Graph-enhanced reasoning networks for explainable robustness scoring in structured question evaluation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月3日

查看全文
  • Olaniyan Deborah,
  • Wario Ruth

摘要

Ensuring that examination questions are pedagogically sound and aligned with the curriculum remains a key challenge in automated assessment. Most existing approaches focus on answer correctness or surface-level textual similarity, overlooking whether questions genuinely reflect syllabus content, require appropriate reasoning, or maintain structural coherence. This study presents SyllabiQA-RSNet, a syllabus-driven framework for automatically evaluating exam question quality. The model combines DeBERTa—large contextual embeddings with graph attention-based syllabus reasoning, capturing dependencies across syllabus sections and multi-step logical connections between questions and course content. Robustness scoring is formulated as a multi-task problem, with the model simultaneously predicting answers, generating reasoning chains, and producing continuous robustness scores. Evaluation on the SyllabusQA benchmark, comprising 5078 question–answer pairs from 63 university syllabi across STEM and non-STEM fields, demonstrates that SyllabiQA-RSNet substantially outperforms baseline models. It achieved a Pearson correlation of 0.79 and Spearman correlation of 0.76 with reference robustness scores, compared to 0.64 for DeBERTa-only and 0.58 for RoBERTa, while reducing RMSE to 0.082. The framework also enhanced answer prediction, particularly for reasoning-intensive questions, improving Exact Match scores by up to 7.2 points and producing higher-quality reasoning chains (+ 8.3 BLEU-4, + 8.8 ROUGE-L). Ablation studies confirmed the necessity of both syllabus graph reasoning and explicit robustness supervision, with their removal causing up to an 11-point drop in robustness correlation. These results indicate that explicitly modeling syllabus structure is critical for reliable, interpretable assessment of question quality, offering insights beyond simple answer correctness and enabling automated evaluation that aligns closely with pedagogical objectives.