Code Refactoring with ChatGPT: Analysis Based on Real and Synthetic Extract Method Opportunities
摘要
Recently, Large Language Models (LLMs) like OpenAI’s ChatGPT have shown potential in various aspects of software engineering. As a result, practitioners are increasingly exploring their use in code refactoring. While prior research has investigated ChatGPT’s capabilities in areas such as requirements engineering, code generation, code review, and unit test generation, its strengths and limitations in code refactoring remain insufficiently explored. This paper evaluates ChatGPT’s capability to identify extract method opportunities (EMOs). Our study leverages two widely used benchmark datasets containing synthetic and real EMOs, previously employed to assess extract method refactoring techniques. The results indicate that while ChatGPT successfully identifies EMOs, its low overlap with benchmark-defined refactorings raises concerns about the practicality of its recommendations. Additionally, while the model excels in generating descriptive method names for the extracted methods, its overall effectiveness requires further validation against multiple refactoring tools.