From Words to Code: Do NLP Prompting Strategies Generalize to Code Generation?
摘要
Prompt engineering plays an important role in optimizing the performance of Large Language Models (LLMs). Although various prompting techniques have achieved substantial gains in natural language processing (NLP), their transferability to code generation remains underexplored. This paper presents an empirical study that examines the effects of NLP-inspired prompting methods on code generation using five LLMs and two established benchmarks, HumanEval and LiveCodeBench. We find that instruction-based prompting produces surprisingly limited or inconsistent improvements in programming contexts, while reasoning-based prompting delivers stronger and more stable performance. Our analysis identified two forms of reasoning-based prompting—procedural and goal-oriented—each addressing different aspects of reasoning and both effective for code generation. Building on these findings, we explore prompt designs that more fully leverage the reasoning capabilities of advanced LLMs, achieving measurable improvements in code generation accuracy. These results emphasize the continuing importance of prompt design in advancing AI-assisted programming.