Transformer-Based Long Time Series Forecasting with Decoupled Information Extraction and Information Complementarity
摘要
In the realm of time series forecasting, the challenge posed by mixed-scale information and inadequate information extraction has long been a vexing problem. At the same time, in order to improve the robustness of the model, the design of the loss function specifically for volatile data is often overlooked but worthy of attention. To address this problem, this paper proposes a novel model consisting of three key parts: Decoupled Information Extraction, Information Complementarity and Optimized Loss Function. To capture the intricate connections within sequences, the correlation across different variable sequences, and the temporal dependencies of seasonal and trend components, three distinct types of attention maps are designed: cross-subsequence attention, cross-variable attention, and cross-time step attention. A channel-independent strategy is employed for the elimination of interaction noise, enabling effective signal-to-noise separation and facilitating decoupled information extraction. The channel-independent strategy eliminates the interaction noise between variables but erases the correlation. By introducing a channel mixing strategy and employing cross-variable attention mechanisms, we are able to effectively preserve the correlation, while achieving the decoupling and complementarity of the correlations. Finally, the results of the three types of attention are fused and input into an MLP for complementary prediction. An optimized loss function is proposed to smooth the volatile loss, significantly reducing the effect on model parameter optimization during training. SOTA forecasting accuracy have been achieved in our experiments.