Test code is a tangible representation of software testing in practice and an important component of the project code. A good summary can help programmers quickly understand test code. However, we find that existing code summarization tasks often overlook the test code. They treat test codes as auto-generated codes and remove them from the dataset. Therefore, the performance of existing summarization model-dataset combinations on test code summarization is unknown. To obtain their performance and find the best model-dataset combination for test code summarization, we conducted an empirical study. The study was built on 5 deep-learning-based models, 3 datasets and 4 metrics. In particular, the 3 datasets include two widely used datasets that don’t contain test code, and the first JUnit Test code summarization dataSet we collected, JTS. The JTS contains 94,270 JUnit test-code-summary pairs extracted from 9,211 Java open source projects on GitHub. Since existing summarization models are already quite robust, while datasets almost all don’t include test code, we modified the datasets in our study. Specifically, in addition to exploring the performance of existing model-dataset combinations on test code summarization, we further investigated the impact of switching to datasets with reintroduced test code and specialized test code summarization datasets on performance. The experimental results show that existing model-dataset combinations are not well-suited for the test code summarization task. And the preferred model-dataset combination solution for this task is \({<}\) pre-trained model, specialized test code summarization dataset \({>}\) .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Empirical Study of Test Code Summarization

  • Yuanyuan Chen,
  • Xiangping Chen,
  • Yuan Huang,
  • Xiaocong Zhou

摘要

Test code is a tangible representation of software testing in practice and an important component of the project code. A good summary can help programmers quickly understand test code. However, we find that existing code summarization tasks often overlook the test code. They treat test codes as auto-generated codes and remove them from the dataset. Therefore, the performance of existing summarization model-dataset combinations on test code summarization is unknown. To obtain their performance and find the best model-dataset combination for test code summarization, we conducted an empirical study. The study was built on 5 deep-learning-based models, 3 datasets and 4 metrics. In particular, the 3 datasets include two widely used datasets that don’t contain test code, and the first JUnit Test code summarization dataSet we collected, JTS. The JTS contains 94,270 JUnit test-code-summary pairs extracted from 9,211 Java open source projects on GitHub. Since existing summarization models are already quite robust, while datasets almost all don’t include test code, we modified the datasets in our study. Specifically, in addition to exploring the performance of existing model-dataset combinations on test code summarization, we further investigated the impact of switching to datasets with reintroduced test code and specialized test code summarization datasets on performance. The experimental results show that existing model-dataset combinations are not well-suited for the test code summarization task. And the preferred model-dataset combination solution for this task is \({<}\) pre-trained model, specialized test code summarization dataset \({>}\) .