Using LLMs to Analyze Software Requirements for Software Testing: A Comparative Study
摘要
The precision and clarity of Software Requirements (SR) are crucial for effective software testing and successful project delivery. Traditional methods of SR analysis, while effective, often demand substantial manual effort and are susceptible to human error. Recent advancements in Natural Language Processing (NLP), particularly with the development of Large Language Models (LLMs), present new opportunities for automating and enhancing the analysis of SRS documents. This paper presents a comparative study on the application of LLMs in analyzing SR for the purpose of improving software testing processes. Our research investigates the capabilities of LLMs in identifying ambiguities, inconsistencies, and gaps in SR documents, which are critical factors influencing the efficacy of software testing. We compare the performance of LLMs against conventional SRS analysis techniques, focusing on key metrics such as precision, recall, and the ability to generate actionable insights. The results of our study indicate that LLMs significantly enhance the accuracy and efficiency of SR analysis. LLMs demonstrated a high precision rate in detecting ambiguous terms and phrases, often suggesting clarifications that align closely with expert recommendations. Moreover, LLMs effectively identified inconsistencies within and across requirement documents, uncovering potential conflicts and redundancies that might be overlooked by traditional methods. These capabilities are particularly beneficial for improving the quality of test cases derived from SR, thereby enhancing the overall software testing process. In addition to performance metrics, the study explores the practical implications of integrating LLM-based analysis into existing software testing workflows. Factors such as scalability, cost-effectiveness, and the learning curve for practitioners are examined. However, the study also acknowledges the challenges and limitations associated with LLM adoption, including issues related to the interpretability of LLM outputs and the necessity for extensive domain-specific training data. Potential solutions and directions for future research are discussed to address these challenges.