Scholarlink

<p>To address the black-box nature, weak interpretability, and low reliability of existing automatic subjective scoring systems for uncertain samples, this paper proposes HMCSDeS3WD, a human–machine collaborative framework that integrates DeBERTa-based semantic encoding, a task-adaptive MLP for ordinal regression, and Sequential Three-Way Decision (S3WD). The DeBERTa encoder extracts deep semantic features from student answers; the MLP maps fused representations into an ordinal scoring space; and a composite confidence metric dynamically routes each sample into one of three paths: automatic machine scoring, human–machine collaborative review, or independent human scoring. Evaluated on the widely adopted ASAP benchmark under the automatic-only mode (no human intervention), the framework achieves an average Quadratic Weighted Kappa (QWK) of 0.862 and a normalized mean absolute error (MAE) of 0.0758, outperforming mainstream baselines including CNN, LSTM, BERT, RoBERTa, DeBERTa, and uncertainty-aware variants. In the full collaborative mode, it reduces manual workload by ≈65% while raising consistency to QWK = 0.91. This work provides a promising solution for building interpretable and reliable intelligent scoring systems in educational assessment.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Research on human–machine collaborative scoring method for subjective questions based on DeBERTa-MLP-S3WD model

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月1日

查看全文
  • Meili Zhu,
  • Wei Wu

摘要

To address the black-box nature, weak interpretability, and low reliability of existing automatic subjective scoring systems for uncertain samples, this paper proposes HMCSDeS3WD, a human–machine collaborative framework that integrates DeBERTa-based semantic encoding, a task-adaptive MLP for ordinal regression, and Sequential Three-Way Decision (S3WD). The DeBERTa encoder extracts deep semantic features from student answers; the MLP maps fused representations into an ordinal scoring space; and a composite confidence metric dynamically routes each sample into one of three paths: automatic machine scoring, human–machine collaborative review, or independent human scoring. Evaluated on the widely adopted ASAP benchmark under the automatic-only mode (no human intervention), the framework achieves an average Quadratic Weighted Kappa (QWK) of 0.862 and a normalized mean absolute error (MAE) of 0.0758, outperforming mainstream baselines including CNN, LSTM, BERT, RoBERTa, DeBERTa, and uncertainty-aware variants. In the full collaborative mode, it reduces manual workload by ≈65% while raising consistency to QWK = 0.91. This work provides a promising solution for building interpretable and reliable intelligent scoring systems in educational assessment.