Bayesian and Box Behnken Design for XGboost Hyperparameters on Energy Consumption Profile Model Optimisation
摘要
Any machine learning model development aims to achieve high accuracy by optimising its hyperparameters using conventional linear, brute force, genetic algorithm and Bayesian techniques. However, most models achieve this optimisation with an acceptable accuracy level at the expense of overfitting or without overfitting but with lower accuracy. Hence, this study aims to employ a novel hyperparameters tuning algorithm with the Bayesian and Box Behnken Design (BBD) of Experiment approach to optimise an XGBoost Energy Consumption Profiling prediction model as the XBBD solution. XBBD is designed to optimise a highly dense time series and heteroscedastic energy consumption profiling data machine learning model to achieve an optimum root means square error (RMSE) with minimal overfitting ratio (OR) for the model. Subsequently, the Exploratory Data Analysis (EDA) determines the outcome of the three hyperparameters based on the best RMSE and minimal overfitting. In the experiment, the EDA identifies an optimised alpha, lambda, and subsample with 3,3,0.45, respectively, with a minimum RMSE of 0.0564 within 33 iterations, 15 trials, and an OR equal to zero. The key finding reveals that the XBBD is able to optimise the model effectively and efficiently through three hyperparameters alpha, lambda and subsample with minimal overfitting and higher accuracy.