Scholarlink

<p>The Adam (Adaptive Moment Estimation) algorithm is widely used in deep learning for its adaptive learning rate and momentum-based gradient updating capabilities. Despite its effectiveness, the original Adam algorithm depends on fixed hyperparameters which limits its flexibility, adaptability and performance. This study addresses these limitations by introducing two refined variants of the Adam algorithm—Algorithm-1 and Algorithm-2 which enhances its adaptability by dynamically tuning the key hyperparameters. Algorithm 1 performs iterative adjustments to the hyperparameters α1, α2 (decay factors) and learning rate η allowing the algorithm to explore their optimal values for a variety of objective functions. Algorithm 2 further fine tunes this approach by focusing on refining the learning rate η within the identified optimal ranges for α1 and α2 obtained from Algorithm-1. These modified algorithms are evaluated on a set of 32 benchmark objective functions and the results obtained evidently shows improvements in the accuracy, time reduction and convergence rate over the standard Adam algorithm. The results infer that adaptive hyperparameter tuning enhances Adam’s optimization effectiveness which can be applied in machine learning and deep learning applications to solve complex problems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Tuning the hyperparameter in ADAM optimizer

  • 期刊论文

  • 发表日期：

    2026年4月6日

查看全文
  • Vigneshwara Raj. A. G.,
  • Punniyamoorthy Murugesan,
  • Nanmaran M

摘要

The Adam (Adaptive Moment Estimation) algorithm is widely used in deep learning for its adaptive learning rate and momentum-based gradient updating capabilities. Despite its effectiveness, the original Adam algorithm depends on fixed hyperparameters which limits its flexibility, adaptability and performance. This study addresses these limitations by introducing two refined variants of the Adam algorithm—Algorithm-1 and Algorithm-2 which enhances its adaptability by dynamically tuning the key hyperparameters. Algorithm 1 performs iterative adjustments to the hyperparameters α1, α2 (decay factors) and learning rate η allowing the algorithm to explore their optimal values for a variety of objective functions. Algorithm 2 further fine tunes this approach by focusing on refining the learning rate η within the identified optimal ranges for α1 and α2 obtained from Algorithm-1. These modified algorithms are evaluated on a set of 32 benchmark objective functions and the results obtained evidently shows improvements in the accuracy, time reduction and convergence rate over the standard Adam algorithm. The results infer that adaptive hyperparameter tuning enhances Adam’s optimization effectiveness which can be applied in machine learning and deep learning applications to solve complex problems.