Hyperparameter optimisation is crucial for maximising machine learning model performance. Still, it is computationally intensive due to the iterative nature of hyperparameter optimisation methods like SMAC, SMBOX, and frameworks like Optuna. We introduce ZeroTune 2.0 (ZT2). This zero-shot hyperparameter optimisation method predicts near-optimal hyperparameters in a single step by leveraging a pretrained model built from a large knowledge base of prior hyperparameter optimisation trials. By integrating with the Optuna framework, incorporating an extensive set dataset meta-parameters, and implementing an automated recursive feature selection, ZT2 improves upon the existing ZeroTune algorithm and significantly reduces computational overhead compared to methods like Bayesian optimisation. Benchmarking shows it outperforms a state-of-the-art iterative optimisation method in the early stages, saving up to five hyperparameter optimisation iterations. These findings demonstrate the effectiveness of ZT2 as an efficient, practical, and adaptable solution for hyperparameter optimisation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ZeroTune 2.0: Enhanced Meta-parameter Selection and Optuna Integration for Zero-Shot Hyperparameter Optimisation

  • Tarek Salhi,
  • John Woodward

摘要

Hyperparameter optimisation is crucial for maximising machine learning model performance. Still, it is computationally intensive due to the iterative nature of hyperparameter optimisation methods like SMAC, SMBOX, and frameworks like Optuna. We introduce ZeroTune 2.0 (ZT2). This zero-shot hyperparameter optimisation method predicts near-optimal hyperparameters in a single step by leveraging a pretrained model built from a large knowledge base of prior hyperparameter optimisation trials. By integrating with the Optuna framework, incorporating an extensive set dataset meta-parameters, and implementing an automated recursive feature selection, ZT2 improves upon the existing ZeroTune algorithm and significantly reduces computational overhead compared to methods like Bayesian optimisation. Benchmarking shows it outperforms a state-of-the-art iterative optimisation method in the early stages, saving up to five hyperparameter optimisation iterations. These findings demonstrate the effectiveness of ZT2 as an efficient, practical, and adaptable solution for hyperparameter optimisation.