Bayesian Model Selection Pruning in Predictive Maintenance
摘要
Deep Neural Network architecture design significantly impacts the final model performance. The process of searching for optimal architectures, known as Neural Architecture Search (NAS), involves training and evaluating an important number of models. Therefore, mechanisms to reduce the resources required for NAS are highly valuable. This work proposes a methodology to prune the Bayesian Optimization process used in NAS. With this aim, an estimator has been trained to predict the future performance of a model by just observing few training and validation epochs. To build such an estimator, the authors developed a dataset containing information (hyperparameters and performance curves) of multiple architectures trained on 62 different predictive maintenance datasets. The results of a simulated BO process used for NAS highlight a reduction in the optimization time of more than 50% with a minimal loss (around 2%) in the performance of the best model found.