Scholarlink

<p>Large language models (LLMs)—advanced AI-powered digital assistants—have achieved notable success in general applications, yet their capabilities in specialized industrial fields such as metal casting remain largely unexamined. To bridge this gap, this study introduces Cast-Bench, a framework that evaluates AI using questions from a national professional qualification exam for foundry engineers. Eleven representative models were systematically tested to determine their accuracy, the role of native Chain-of-Thought (CoT) mechanisms, and the influence of language. The results reveal a stratified distribution of knowledge, with the best-performing systems achieving accuracies above 85% but demonstrating competence closer to novice practitioners than domain experts. The effectiveness of CoT was found to vary by architecture, while several models exhibited pronounced linguistic bias. These findings provide the first quantitative assessment of LLM performance in metal casting and indicate that, although current systems may assist in supporting technical tasks, their deployment in practice requires careful expert oversight.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Assessing the Proficiency of Large Language Models in the Metal Casting: An Empirical Study

  • 期刊论文

  • 发表日期：

    2026年2月23日

查看全文
  • Han Yu,
  • Zheng Yin,
  • Zhida Jin,
  • Dorota Wilk-Kołodziejczyk,
  • Shuang Duan,
  • Shuo Yu,
  • Yichen Huo,
  • Chunling Bao

摘要

Large language models (LLMs)—advanced AI-powered digital assistants—have achieved notable success in general applications, yet their capabilities in specialized industrial fields such as metal casting remain largely unexamined. To bridge this gap, this study introduces Cast-Bench, a framework that evaluates AI using questions from a national professional qualification exam for foundry engineers. Eleven representative models were systematically tested to determine their accuracy, the role of native Chain-of-Thought (CoT) mechanisms, and the influence of language. The results reveal a stratified distribution of knowledge, with the best-performing systems achieving accuracies above 85% but demonstrating competence closer to novice practitioners than domain experts. The effectiveness of CoT was found to vary by architecture, while several models exhibited pronounced linguistic bias. These findings provide the first quantitative assessment of LLM performance in metal casting and indicate that, although current systems may assist in supporting technical tasks, their deployment in practice requires careful expert oversight.