Scholarlink

<p>Classifying code snippet-based questions is essential for teaching, preparing assessment materials, and supporting intelligent learning systems in programming education. Traditional frequency-based encodings, such as TF-IDF, often fail to capture the contextual semantics within code-related questions. This study employs contextualized embeddings generated by the large language model Text-Embedding-3-Large (TE3L) to evaluate their effectiveness in classifying code-related questions. It further investigates which classifier architecture best complements the TE3L representation. Using a small-scale dataset of 171 SQL certification-style questions representative of course-level repositories, we analyze the classification complexity reduced by the TE3L scheme compared to TF-IDF. Then, we investigate classification performance under various classifier architectures with TE3L embeddings, including single models, boosting, and stacking ensembles. Results demonstrate that the TE3L scheme significantly reduces classification complexity and improves performance compared to the TF-IDF. Single classifiers, particularly the support vector machine with a linear kernel and the stochastic gradient descent classifiers, performed the best with the TE3L scheme and achieved an 11-percentage-point relative improvement over the benchmark in the weighted macro-average F1 score. The boosting and stacking techniques did not enhance performance, reflecting the challenges of ensemble learning under small-sample, imbalanced conditions. This work highlights the practical value of using LLM-based embeddings to automate question classification in low-resource educational contexts, supporting teachers in building intelligent assessment tools without requiring deep expertise in NLP or machine learning.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A practical approach to classifying code-snippet questions in small-scale educational repositories using LLM embeddings

  • 期刊论文

  • 发表日期：

    2026年4月24日

查看全文
  • Hung-Yi Chen,
  • Ying-Chieh Liu,
  • Po-Chou Shih,
  • Tiffany Chiu,
  • Tzong-Ming Cheng

摘要

Classifying code snippet-based questions is essential for teaching, preparing assessment materials, and supporting intelligent learning systems in programming education. Traditional frequency-based encodings, such as TF-IDF, often fail to capture the contextual semantics within code-related questions. This study employs contextualized embeddings generated by the large language model Text-Embedding-3-Large (TE3L) to evaluate their effectiveness in classifying code-related questions. It further investigates which classifier architecture best complements the TE3L representation. Using a small-scale dataset of 171 SQL certification-style questions representative of course-level repositories, we analyze the classification complexity reduced by the TE3L scheme compared to TF-IDF. Then, we investigate classification performance under various classifier architectures with TE3L embeddings, including single models, boosting, and stacking ensembles. Results demonstrate that the TE3L scheme significantly reduces classification complexity and improves performance compared to the TF-IDF. Single classifiers, particularly the support vector machine with a linear kernel and the stochastic gradient descent classifiers, performed the best with the TE3L scheme and achieved an 11-percentage-point relative improvement over the benchmark in the weighted macro-average F1 score. The boosting and stacking techniques did not enhance performance, reflecting the challenges of ensemble learning under small-sample, imbalanced conditions. This work highlights the practical value of using LLM-based embeddings to automate question classification in low-resource educational contexts, supporting teachers in building intelligent assessment tools without requiring deep expertise in NLP or machine learning.