Automated Test Case Generation from Natural Language Requirements Using Large Language Models
摘要
Test case generation is a critical aspect of software testing that ensures software systems function correctly under various conditions. As software complexity increases, traditional manual test case creation becomes inefficient, necessitating automated approaches. This study investigates the potential of Large Language Models (LLMs), specifically Claude 3.5 Sonnet, in automating the extraction of conditional statements from natural language (NL) functional requirements—a fundamental step in automated test case derivation. Prompt Engineering techniques are evaluated, including zero-shot, few-shot, and chain-of-thought prompting, for extracting antecedents and consequents, assessing performance using a weighted similarity metric that integrates BERT-Score, BLEU, and ROUGE. Experiments on the PURE dataset show that few-shot prompting achieves the best F1 score, balancing accuracy and efficiency. Beyond performance evaluation, we examine generalizability across requirement types, comparison with manual annotations, and computational trade-offs. We also identify key challenges, including handling unmarked conditionals and complex logical structures. These findings highlight both the potential and limitations of LLMs in test case automation, offering deeper insights into their role in software engineering workflows.