Scholarlink

<p>Automated program repair (APR) is proposed to reduce manual debugging efforts by automatically fixing buggy programs. Traditional APR techniques rely heavily on test cases, categorized into trigger-based and trigger-free approaches. While trigger-based methods achieve higher accuracy, their dependence on well-established test suites limits real-world applicability. Trigger-free methods, though more flexible, suffer from inferior fault localization performance due to the absence of trigger tests. Recent advances leverage large language models (LLMs) to generate bug-reproducing test cases, yet their systematic integration across the APR pipeline remains unexplored. This paper presents the first comprehensive study on leveraging LLM-generated tests throughout the APR workflow. We conduct experiments on 374 single-function bugs from Defects4J, systematically evaluating the impact of LLM-generated tests on fault localization, patch generation, and patch validation. Key findings reveal that: (1) Even LLM-generated tests with incorrect assertions can enhance fault localization for trigger-free APR by providing supplementary execution traces, improving Top@1 bug detection by 61% ; (2) Incorporating these tests into patch ranking boosts repair effectiveness by 12.5%–24.6% across Top@1–Top@5 metrics. Then we propose a novel APR framework <span>GT-Repair</span> which incorporates LLM-generated tests into different repair stages. Compared to the trigger-free APR pipeline without LLM-generated tests, <span>GT-Repair</span> achieves a 38.8% improvement in Top@1 repair performance. Besides, <span>GT-Repair</span> achieves state-of-the-art performance on Defects4J in trigger-free scenarios, compared to state-of-the-art test-based approaches. This work demonstrates the feasibility of LLM-generated tests in APR, and provides actionable insights for future works.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Can test cases generated by large language models facilitate automated program repair?

  • 期刊论文

  • 发表日期：

    2026年2月7日

查看全文
  • Chengming Zhang,
  • Haoye Wang,
  • Chuyang Xu,
  • Jiakun Liu,
  • Kui Liu,
  • Zhongxin Liu

摘要

Automated program repair (APR) is proposed to reduce manual debugging efforts by automatically fixing buggy programs. Traditional APR techniques rely heavily on test cases, categorized into trigger-based and trigger-free approaches. While trigger-based methods achieve higher accuracy, their dependence on well-established test suites limits real-world applicability. Trigger-free methods, though more flexible, suffer from inferior fault localization performance due to the absence of trigger tests. Recent advances leverage large language models (LLMs) to generate bug-reproducing test cases, yet their systematic integration across the APR pipeline remains unexplored. This paper presents the first comprehensive study on leveraging LLM-generated tests throughout the APR workflow. We conduct experiments on 374 single-function bugs from Defects4J, systematically evaluating the impact of LLM-generated tests on fault localization, patch generation, and patch validation. Key findings reveal that: (1) Even LLM-generated tests with incorrect assertions can enhance fault localization for trigger-free APR by providing supplementary execution traces, improving Top@1 bug detection by 61% ; (2) Incorporating these tests into patch ranking boosts repair effectiveness by 12.5%–24.6% across Top@1–Top@5 metrics. Then we propose a novel APR framework GT-Repair which incorporates LLM-generated tests into different repair stages. Compared to the trigger-free APR pipeline without LLM-generated tests, GT-Repair achieves a 38.8% improvement in Top@1 repair performance. Besides, GT-Repair achieves state-of-the-art performance on Defects4J in trigger-free scenarios, compared to state-of-the-art test-based approaches. This work demonstrates the feasibility of LLM-generated tests in APR, and provides actionable insights for future works.