Test case generation is a critical aspect of software testing that ensures software systems function correctly under various conditions. As software complexity increases, traditional manual test case creation becomes inefficient, necessitating automated approaches. This study investigates the potential of Large Language Models (LLMs), specifically Claude 3.5 Sonnet, in automating the extraction of conditional statements from natural language (NL) functional requirements—a fundamental step in automated test case derivation. Prompt Engineering techniques are evaluated, including zero-shot, few-shot, and chain-of-thought prompting, for extracting antecedents and consequents, assessing performance using a weighted similarity metric that integrates BERT-Score, BLEU, and ROUGE. Experiments on the PURE dataset show that few-shot prompting achieves the best F1 score, balancing accuracy and efficiency. Beyond performance evaluation, we examine generalizability across requirement types, comparison with manual annotations, and computational trade-offs. We also identify key challenges, including handling unmarked conditionals and complex logical structures. These findings highlight both the potential and limitations of LLMs in test case automation, offering deeper insights into their role in software engineering workflows.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automated Test Case Generation from Natural Language Requirements Using Large Language Models

  • Ouafae Karmouda,
  • Maroua Ghaouat,
  • Guillaume Feuilloley

摘要

Test case generation is a critical aspect of software testing that ensures software systems function correctly under various conditions. As software complexity increases, traditional manual test case creation becomes inefficient, necessitating automated approaches. This study investigates the potential of Large Language Models (LLMs), specifically Claude 3.5 Sonnet, in automating the extraction of conditional statements from natural language (NL) functional requirements—a fundamental step in automated test case derivation. Prompt Engineering techniques are evaluated, including zero-shot, few-shot, and chain-of-thought prompting, for extracting antecedents and consequents, assessing performance using a weighted similarity metric that integrates BERT-Score, BLEU, and ROUGE. Experiments on the PURE dataset show that few-shot prompting achieves the best F1 score, balancing accuracy and efficiency. Beyond performance evaluation, we examine generalizability across requirement types, comparison with manual annotations, and computational trade-offs. We also identify key challenges, including handling unmarked conditionals and complex logical structures. These findings highlight both the potential and limitations of LLMs in test case automation, offering deeper insights into their role in software engineering workflows.