Software testing regularly involves numerous setups and user inputs, leading to a combinatorial explosion of test cases. While Combinatorial Interaction Testing (CIT) has been theoretically investigated, its effectiveness in real-world scenarios remains unclear [1]. This research fills that gap by utilizing CIT in some live software projects. We led two studies: the first focuses on optimizing user input testing in jTrac, and the second focused on managing system configurations in Redmine, a comparative web application. We looked at CIT to customary testing strategies, breaking down components like test design time, test automation, test execution, suite size, and defect detection. The investigation gave valuable insights into enhancing CIT execution and reception. The results are promising. With CIT, the number of required test cases is significantly reduced, but at the same time, defect detection is improved. In the first study, the average time to detect a defect was 1.40 h (design, automation, execution, and evaluation) compared to 0.35 h with CIT. Similar patterns emerged in the second study. These findings have important implications for both researchers and organizations. They highlight CIT’s promise for software testing, including decreasing test case burden and perhaps improving defect detection rates. This study provides practical evidence for organizations and testers looking to improve their testing procedures.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating the Impact of Combinatorial Interaction Testing on Test Automation: A Case Study from Industry

  • Feras Daoud,
  • Miroslav Bures,
  • Zdenek David,
  • Petr Syrovatka

摘要

Software testing regularly involves numerous setups and user inputs, leading to a combinatorial explosion of test cases. While Combinatorial Interaction Testing (CIT) has been theoretically investigated, its effectiveness in real-world scenarios remains unclear [1]. This research fills that gap by utilizing CIT in some live software projects. We led two studies: the first focuses on optimizing user input testing in jTrac, and the second focused on managing system configurations in Redmine, a comparative web application. We looked at CIT to customary testing strategies, breaking down components like test design time, test automation, test execution, suite size, and defect detection. The investigation gave valuable insights into enhancing CIT execution and reception. The results are promising. With CIT, the number of required test cases is significantly reduced, but at the same time, defect detection is improved. In the first study, the average time to detect a defect was 1.40 h (design, automation, execution, and evaluation) compared to 0.35 h with CIT. Similar patterns emerged in the second study. These findings have important implications for both researchers and organizations. They highlight CIT’s promise for software testing, including decreasing test case burden and perhaps improving defect detection rates. This study provides practical evidence for organizations and testers looking to improve their testing procedures.