Enhancing Test Case Prioritization in Continuous Integration with MixTCP: A Replication Study Across Multiple Projects
摘要
The rapid advancements in software development call for efficient and effective testing processes, particularly within Continuous Integration (CI) environments. Test Case Prioritization (TCP) is critical for early fault detection, but traditional methods often fall short in dynamic settings. In our previous work, we introduced and validated the MixTCP tool, which integrates the NEUTRON neural network model for TCP, by conducting experiments in a private industrial project involving 59 test files and 297 test cases. Building on these findings, this paper extends the validation of the MixTCP tool to new industrial contexts. We conducted a replication study using additional datasets from three open-source projects: Finch, Ash Paper Trail, and Phoenix, involving varying numbers of test cases and CI cycles. These experiments aimed to test the scalability of the tool, its effectiveness, and its practical applicability across different environments. Our extended study confirms that MixTCP consistently improves the NAPFD metric compared to the usual process (execution of test cases in random order) across various datasets and conditions. Additionally, this study reinforces the previously shown benefits of MixTCP, including its efficiency, user-friendliness, and ease of integration into existing workflows. The architecture of MixTCP, with its loosely coupled components (Mix TCP task, TCP Server, and NEUTRON model), allows for seamless integration into diverse development environments. By validating MixTCP across various industrial scenarios, this study reinforces its value as a robust solution for modern software testing challenges.