Deep Neural Network architecture design significantly impacts the final model performance. The process of searching for optimal architectures, known as Neural Architecture Search (NAS), involves training and evaluating an important number of models. Therefore, mechanisms to reduce the resources required for NAS are highly valuable. This work proposes a methodology to prune the Bayesian Optimization process used in NAS. With this aim, an estimator has been trained to predict the future performance of a model by just observing few training and validation epochs. To build such an estimator, the authors developed a dataset containing information (hyperparameters and performance curves) of multiple architectures trained on 62 different predictive maintenance datasets. The results of a simulated BO process used for NAS highlight a reduction in the optimization time of more than 50% with a minimal loss (around 2%) in the performance of the best model found.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Bayesian Model Selection Pruning in Predictive Maintenance

  • David Solis-Martin,
  • Juan Galan-Paez,
  • Joaquin Borrego-Diaz

摘要

Deep Neural Network architecture design significantly impacts the final model performance. The process of searching for optimal architectures, known as Neural Architecture Search (NAS), involves training and evaluating an important number of models. Therefore, mechanisms to reduce the resources required for NAS are highly valuable. This work proposes a methodology to prune the Bayesian Optimization process used in NAS. With this aim, an estimator has been trained to predict the future performance of a model by just observing few training and validation epochs. To build such an estimator, the authors developed a dataset containing information (hyperparameters and performance curves) of multiple architectures trained on 62 different predictive maintenance datasets. The results of a simulated BO process used for NAS highlight a reduction in the optimization time of more than 50% with a minimal loss (around 2%) in the performance of the best model found.