Exploration Study About LLM with Proper Prompt in Automated Program Repair
摘要
Automated Program Repair (APR) is currently receiving increasing attention in the field of software engineering because more and more bugs are emerging with the continuous expansion of program size. Recently, Large Language Models (LLMs) have got an outstanding progress in automated debugging field. Some researches utilized Language Models that pre-trained on source code, such as UnixCoder, Incoder, etc., to generate patches for bugs, and achieved impressive performance. Noteworthy, large language models are proved to benefit from in-context learning (ICL). By receiving well designed prompt, models can understand user needs better and provide a more accurate response. However, we find that existing prompted LLM-based APR methods either used too simple prompts or too complex ones. Therefore, this study explores and validates the proper structure of prompt that helps LLM fix more bugs. In this paper, we present a medium length prompt format to enhance CodeLlama’s repairing capability. Specifically, we design prompt that include three part: role-profile, bug informations and repair instruction. For each bug, we extract their corresponding informations such as function description and bug location, fill in the prompt frame, and finally send it to CodeLlama to generate a patch. Our experiments show that the proper length of prompt can help CodeLlama model fix more bugs correctly.