Scholarlink

With the rapid advancement of deep learning, neural networks have demonstrated remarkable potential across various fields. While traditional gradient descent methods offer fast convergence, they often encounter challenges such as getting trapped in local minima. On the other hand, evolutionary algorithms excel in global search capabilities but are hindered by prolonged training times. As a result, optimizing large-scale neural networks under limited computational resources remains a pressing challenge. Therefore, this paper proposes the Adaptive Gradient-directed Differential Evolution (AGDE) algorithm, which combines the strengths of differential evolution and first-order optimization algorithms. AGDE utilizes first-order gradient information to guide the update direction of the differential mutation operator, effectively avoiding poor local optima, identifying superior solutions, and accelerating network convergence. Experimental results demonstrate that AGDE significantly enhances classification performance in neural network training. Compared to the best Multi-Gradient Descent algorithms, AGDE consistently outperforms all baselines in ResNet18, achieving an average accuracy improvement of 0.48% over the second-best algorithms, surpassing MAdam by 1.42% and MAdamW by 0.93%. The code is available at https://github.com/YuuuuQ/AGDE .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An Adaptive Gradient-Directed Differential Evolution Algorithm for Training Neural Networks

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Qi Yu,
  • Hang He,
  • Jinmeng Liu,
  • Xijun Liang

摘要

With the rapid advancement of deep learning, neural networks have demonstrated remarkable potential across various fields. While traditional gradient descent methods offer fast convergence, they often encounter challenges such as getting trapped in local minima. On the other hand, evolutionary algorithms excel in global search capabilities but are hindered by prolonged training times. As a result, optimizing large-scale neural networks under limited computational resources remains a pressing challenge. Therefore, this paper proposes the Adaptive Gradient-directed Differential Evolution (AGDE) algorithm, which combines the strengths of differential evolution and first-order optimization algorithms. AGDE utilizes first-order gradient information to guide the update direction of the differential mutation operator, effectively avoiding poor local optima, identifying superior solutions, and accelerating network convergence. Experimental results demonstrate that AGDE significantly enhances classification performance in neural network training. Compared to the best Multi-Gradient Descent algorithms, AGDE consistently outperforms all baselines in ResNet18, achieving an average accuracy improvement of 0.48% over the second-best algorithms, surpassing MAdam by 1.42% and MAdamW by 0.93%. The code is available at https://github.com/YuuuuQ/AGDE .