Scholarlink

<p>Test managers create test plans as blueprints for effective software testing, but manual generation is time-consuming and requires expertise. While Large Language Models (LLMs) have been investigated for test-case generation in prior studies, their application to automated test plan creation remains limited and underexplored. This study investigates the use of LLMs to automate test plan generation in software testing, with a focus on compliance with established standards and customization for project-specific needs. It aims to explore not only how to test but also what to test. Test plans were generated for three open-source web applications using five different LLMs, namely GPT-4o, Cohere, Mistral, Llama3.1, and Google Gemini 1.5. Using the LangChain framework, each LLM was prompted to generate test plans. The generated plans were evaluated for standard conformity, practical feasibility, estimated testing effort, and readability. The test cases derived from the main functionalities were organized into suites in the test management tool <i>TestLink</i>, executed against the applications under test, and the outcomes were recorded as execution reports, which were extracted from the TestLink database for evaluation. From the evaluated open-source applications, it was observed that the quality of the generated test plans varied across different models, with real-world applications performing relatively better than the demo-based sample application in this empirical setting. While LLMs demonstrate potential in automating test planning, human validation is still necessary. LLMs can support test management by increasing automation and efficiency while requiring human oversight to ensure accuracy and reliability.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Automated test plan generation using large language models

  • 期刊论文

  • 发表日期：

    2026年2月21日

查看全文
  • Susmita Haldar,
  • Luiz Fernando Capretz

摘要

Test managers create test plans as blueprints for effective software testing, but manual generation is time-consuming and requires expertise. While Large Language Models (LLMs) have been investigated for test-case generation in prior studies, their application to automated test plan creation remains limited and underexplored. This study investigates the use of LLMs to automate test plan generation in software testing, with a focus on compliance with established standards and customization for project-specific needs. It aims to explore not only how to test but also what to test. Test plans were generated for three open-source web applications using five different LLMs, namely GPT-4o, Cohere, Mistral, Llama3.1, and Google Gemini 1.5. Using the LangChain framework, each LLM was prompted to generate test plans. The generated plans were evaluated for standard conformity, practical feasibility, estimated testing effort, and readability. The test cases derived from the main functionalities were organized into suites in the test management tool TestLink, executed against the applications under test, and the outcomes were recorded as execution reports, which were extracted from the TestLink database for evaluation. From the evaluated open-source applications, it was observed that the quality of the generated test plans varied across different models, with real-world applications performing relatively better than the demo-based sample application in this empirical setting. While LLMs demonstrate potential in automating test planning, human validation is still necessary. LLMs can support test management by increasing automation and efficiency while requiring human oversight to ensure accuracy and reliability.