Scholarlink

<p>Practical hyperparameter tuning is vital for maximizing the performance of machine learning algorithms, as well-tuned hyperparameters are central to advancements in the field. However, the high computational demands of exploring hyperparameters in complex models and large datasets pose significant limitations, often making the tuning process inefficient. Moreover, with the rise of online machine learning applications, the need for rapid response generation has intensified, making algorithm acceleration and data efficiency critical priorities. Meta-learning, or “learning to learn,” offers a promising solution to these challenges by enabling rapid adaptation to new tasks through meta-knowledge from various related tasks. This study proposes a hyperparameter optimization method that combines Bayesian optimization and meta-learning. Our approach leverages prior task knowledge, using similarity measurements derived from meta-features to suggest initial hyperparameters for warm-starting. As the optimization process unfolds, the algorithm dynamically alters similarity measurements to model evaluation techniques, facilitating the generation of effective early responses and continually improving outcomes. With additional evaluations and the development of the surrogate model, the algorithm shifts toward more data-driven Bayesian modeling on the target task, enhancing the quality of the final responses. Moreover, to optimize resource use, the proposed method incorporates a Bayesian learning-curve early-stopping rule that limits resource expenditure on low-potential configurations while avoiding optimistic bias via censoring. Experimental results show that the proposed method delivers, on average, 29.99% better initial performance compared to non-transfer approaches. The final results are also improved, achieving an average accuracy gain of 4.17% over these competitors. Against meta-learning-based methods, the algorithm provides an average improvement of 15.5% in the initial solution and 5.22% in the final solution.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Dynamic meta-learning acquisition function method for Bayesian optimization with early stopping criteria for hyperparameter optimization

  • 期刊论文

  • 发表日期：

    2026年2月13日

查看全文
  • Farshad Seifi,
  • Seyed Taghi Akhavan Niaki

摘要

Practical hyperparameter tuning is vital for maximizing the performance of machine learning algorithms, as well-tuned hyperparameters are central to advancements in the field. However, the high computational demands of exploring hyperparameters in complex models and large datasets pose significant limitations, often making the tuning process inefficient. Moreover, with the rise of online machine learning applications, the need for rapid response generation has intensified, making algorithm acceleration and data efficiency critical priorities. Meta-learning, or “learning to learn,” offers a promising solution to these challenges by enabling rapid adaptation to new tasks through meta-knowledge from various related tasks. This study proposes a hyperparameter optimization method that combines Bayesian optimization and meta-learning. Our approach leverages prior task knowledge, using similarity measurements derived from meta-features to suggest initial hyperparameters for warm-starting. As the optimization process unfolds, the algorithm dynamically alters similarity measurements to model evaluation techniques, facilitating the generation of effective early responses and continually improving outcomes. With additional evaluations and the development of the surrogate model, the algorithm shifts toward more data-driven Bayesian modeling on the target task, enhancing the quality of the final responses. Moreover, to optimize resource use, the proposed method incorporates a Bayesian learning-curve early-stopping rule that limits resource expenditure on low-potential configurations while avoiding optimistic bias via censoring. Experimental results show that the proposed method delivers, on average, 29.99% better initial performance compared to non-transfer approaches. The final results are also improved, achieving an average accuracy gain of 4.17% over these competitors. Against meta-learning-based methods, the algorithm provides an average improvement of 15.5% in the initial solution and 5.22% in the final solution.