Two-Stage Tuning and Bayesian Optimization for the edRVFL Network
摘要
In this paper, we introduce two new tuning strategies for the Ensemble Deep Random Vector Functional Link Network (edRVFL): two-stage tuning (edRVFL-TS) and Bayesian optimization (edRVFL-BO). The two-stage tuning approach simplifies the traditional grid search by dividing it into two phases: rough-tuning and fine-tuning. In the first stage, we identify approximate hyperparameter values, which are then refined to optimal values in the second stage. Bayesian optimization, on the other hand, is a sequential search method that leverages a probabilistic model and an acquisition function to optimize black-box problems. This approach enables the adjustment of continuous variables, allowing for more precise regularization parameters for the edRVFL network. Experiments conducted on UCI datasets demonstrate that edRVFL-BO delivers the best overall performance. Additionally, a comparison of tuning times shows that both edRVFL-TS and edRVFL-BO are faster than the original edRVFL network in finding optimal hyperparameters. Notably, edRVFL-BO proves to be the most efficient among these models, requiring only about 50% of the tuning time needed by the original method.