Scholarlink

Short-term forecasting is important in real-world applications like financial market analysis and emergency response planning, enabling timely decisions and rapid adjustments to minimize risks and improve operational efficiency. Recent studies indicate that while many Transformer-based methods have successfully extended the prediction time horizon, the improvements is rather limited in the accuracy of short-term forecasts, coupled with increasing complexity of these models. To address these challenges, we present SplineFormer in this paper, a novel model that incorporates Kolmogorov-Arnold Networks (KAN) into Transformer and enhance the ProbSparse Self-Attention mechanism by introducing Value into the Query selection process. This study is the first systematic investigation of leveraging KAN into various layers of a Transformer model. Extensive experimental results show that our SplineFormer with replacing the Decoder’s MLP layer with KAN achieves the best results in comparison with other state-of-the-art approaches. We also provide empirical analysis suggesting that positioning KAN near the output layer enhances gradient flow stability and mitigates vanishing or exploding gradients. These findings demonstrate the efficacy of SplineFormer in short-term forecasting tasks and offer valuable insights into effectively integrating KAN for improved model performance.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SplineFormer: Improving Time Series Forecasting with Kolmogorov-Arnold Networks and Enhanced ProbSparse Self-Attention

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Qingwen Zeng,
  • Nan Yang,
  • Jushang Qiu,
  • Junbin Gao,
  • Huaming Chen

摘要

Short-term forecasting is important in real-world applications like financial market analysis and emergency response planning, enabling timely decisions and rapid adjustments to minimize risks and improve operational efficiency. Recent studies indicate that while many Transformer-based methods have successfully extended the prediction time horizon, the improvements is rather limited in the accuracy of short-term forecasts, coupled with increasing complexity of these models. To address these challenges, we present SplineFormer in this paper, a novel model that incorporates Kolmogorov-Arnold Networks (KAN) into Transformer and enhance the ProbSparse Self-Attention mechanism by introducing Value into the Query selection process. This study is the first systematic investigation of leveraging KAN into various layers of a Transformer model. Extensive experimental results show that our SplineFormer with replacing the Decoder’s MLP layer with KAN achieves the best results in comparison with other state-of-the-art approaches. We also provide empirical analysis suggesting that positioning KAN near the output layer enhances gradient flow stability and mitigates vanishing or exploding gradients. These findings demonstrate the efficacy of SplineFormer in short-term forecasting tasks and offer valuable insights into effectively integrating KAN for improved model performance.