Large language models (LLMs) are easy for students to access via off-the-shelf Generative AI tools such as ChatGPT-3.5 and ChatGPT-4o. The ability of these tools to effectively translate natural language into code has forced educators to rethink assessments in the context of Computer Science courses. In this study, we examine whether such tools can generate code in Verilog for circuit design problems. We analyse 153 Verilog questions from the instructional website HDLBits. These problems range from simple combinational circuits to complex sequential circuits. Our results indicate that off-the-shelf tools can often generate code successfully in their first attempts. Further, when the resulting code fails, these tools can often rectify errors by using error and warning messages from the HDLBits simulation results, with little or no additional human input. Thus, our results suggest that educators may need to rethink assessments involving circuit design problems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

How Effectively Can Off-the-Shelf Generative AI Tools Solve Circuit Design Problems?

  • Md Azhaan,
  • Mohd Afeef,
  • Viraj Kumar

摘要

Large language models (LLMs) are easy for students to access via off-the-shelf Generative AI tools such as ChatGPT-3.5 and ChatGPT-4o. The ability of these tools to effectively translate natural language into code has forced educators to rethink assessments in the context of Computer Science courses. In this study, we examine whether such tools can generate code in Verilog for circuit design problems. We analyse 153 Verilog questions from the instructional website HDLBits. These problems range from simple combinational circuits to complex sequential circuits. Our results indicate that off-the-shelf tools can often generate code successfully in their first attempts. Further, when the resulting code fails, these tools can often rectify errors by using error and warning messages from the HDLBits simulation results, with little or no additional human input. Thus, our results suggest that educators may need to rethink assessments involving circuit design problems.