Diagnosing and Repairing Field Access Hallucinations in LLM-Based Test Generation
摘要
Large language models (LLMs) are increasingly used to generate unit tests directly from source code. However, their generated test cases often fail due to hallucinated references to non-existent class fields, a phenomenon we term Field Access Hallucination (FAH). Despite its frequency and impact on test executability, FAH has not been systematically studied in prior work. This paper presents the first framework that uses static guidance to diagnose and correct FAH in LLM-generated test code. We begin by analyzing the prevalence and patterns of FAH errors under minimal prompt settings, revealing that they account for a substantial proportion of test failures. To mitigate FAH, we propose a repair pipeline that combines static analysis, hallucination detection, and prompt rewriting. Our approach statically analyzes attribute access paths in the target function and selectively injects only the necessary field initialization logic into the prompt, improving test reliability without overwhelming the LLM with irrelevant context. We evaluate our method on seven real-world Python modules spanning multiple projects. Experimental results show that our approach increases test execution success rates compared to naive prompting and outperforms full-context prompting strategies in mitigating FAH. Furthermore, our method maintains prompt conciseness while achieving targeted error reduction. These findings suggest that addressing FAH is essential for practical and scalable LLM-based test generation, and that static guidance offers an effective path forward.