Scholarlink

<p>Source code optimization enables developers to enhance programs at the human–computer interaction level, thereby improving development efficiency and product quality. With the rise of large language models (LLMs), fine-tuning and prompting have become mainstream solutions for this task. However, both approaches present challenges: fine-tuning is resource-intensive due to the exponential growth in the scale of LLMs, whereas prompting, although resource-efficient, struggles to generate high-quality optimized programs. In this paper, we present <span>CodeOPT</span>, a LoRA-driven approach for fine-tuning LLMs to optimize C/C++ code. Instead of fine-tuning all LLM parameters, <span>CodeOPT</span> leverages LoRA to fine-tune only an optimization adapter, significantly reducing the number of trainable parameters. Additionally, we incorporate prior optimization knowledge during fine-tuning and introduce optimization-based instruction fine-tuning, enabling LLMs to effectively learn from external knowledge sources to improve program optimization. To evaluate the effectiveness of <span>CodeOPT</span>, we benchmarked it against several baselines on challenging programming tasks from different code completion platforms. Experimental results demonstrate that <span>CodeOPT</span> outperforms all baselines, including the state of the art, while keeping modifications to the original program minimal.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Efficient program optimization through knowledge-enhanced LoRA fine-tuning of large language models

  • 期刊论文

  • 发表日期：

    2025年6月10日

查看全文
  • Caixu Xu,
  • Hui Guo,
  • Caicun Cen,
  • Minglang Chen,
  • Xiongjie Tao,
  • Jie He

摘要

Source code optimization enables developers to enhance programs at the human–computer interaction level, thereby improving development efficiency and product quality. With the rise of large language models (LLMs), fine-tuning and prompting have become mainstream solutions for this task. However, both approaches present challenges: fine-tuning is resource-intensive due to the exponential growth in the scale of LLMs, whereas prompting, although resource-efficient, struggles to generate high-quality optimized programs. In this paper, we present CodeOPT, a LoRA-driven approach for fine-tuning LLMs to optimize C/C++ code. Instead of fine-tuning all LLM parameters, CodeOPT leverages LoRA to fine-tune only an optimization adapter, significantly reducing the number of trainable parameters. Additionally, we incorporate prior optimization knowledge during fine-tuning and introduce optimization-based instruction fine-tuning, enabling LLMs to effectively learn from external knowledge sources to improve program optimization. To evaluate the effectiveness of CodeOPT, we benchmarked it against several baselines on challenging programming tasks from different code completion platforms. Experimental results demonstrate that CodeOPT outperforms all baselines, including the state of the art, while keeping modifications to the original program minimal.