Two-Stage Trained Stacking Model for Univariate Time Series Forecasting
摘要
The stacking ensemble model is widely used in the forecasting of univariate time series data. It works by combining the predictions of multiple models. It has been applied across fields such as economics, energy, and healthcare, where data often fluctuates frequently and comes in diverse forms. First, a set of base models is trained on the dataset to make initial predictions. These predictions are then used as input features for the training of a meta-model. Finally, in subsequent forecasts, the trained meta-model merges the new predictions of the base models to provide a more accurate forecast. However, most stacking models directly use all available data to train the base models once and stack their predictions to train the meta-model. This may lead to overfitting because they train the base models on the entire dataset, including the part of the actual labels for training the meta-model, potentially causing target leakage for the meta-model. To address this issue, we propose a two-stage trained stacking model. The input data is divided into training and label parts. In the first stage, the base models are trained on the training part, and the predictions of the base models are combined with the label part to train the meta-model. In the second stage, the base models are retrained with all input data, and the meta-model trained in the first stage is used for the final prediction. This approach helps mitigate overfitting in the prediction phase caused by target leakage during the training process. We test our model on three different types of datasets. Experimental results show that our stacking ensemble model outperforms the individual base models on all datasets in terms of MAE and MSE, demonstrating not only good generalizability but also improved performance across various scenarios. Additionally, we compared our two-stage trained stacking model with a basic stacking ensemble model framework. The results suggest our model provides more accurate predictions for datasets without clear seasonal features. The code is available at https://github.com/HaiMianXiongDi/Two-Stage-trained-Stacking-Model .