Time series forecasting has been widely applied across multiple domains including databases, finance and transportation. While Transformer-based models have made significant progress in capturing long-term dependencies through attention mechanisms, they struggle with modeling mixed multi-scale periodicity and dynamic trends. In this study, we propose PAPT, a Transformer-based general framework to capture multi-periodicity and long-term trends in time series. On one hand, we introduce Periodic Attention Positional Encoding (PAPE), which incorporates multiple periodic patterns into the attention encoding to enhance weight correlations at periodic synchronization points in the attention matrix. Furthermore, we employ a two-layer feature generation module (TGM) to capture fluctuations within cycles. On the other hand, we design an adaptive polynomial basis trend fitting module (PTM) that learns trend features through weight adjustment and fits them with multiple distinct polynomial bases. Compared with 8 baselines, PAPT demonstrates state-of-the-art forecasting performance through extensive experiments conducted on 10 real-world datasets in 4 domains. The code and datasets are shown in https://github.com/wxy1225/Periodic-Attention-with-Polynomial-Trend-Fitting-Transformer.git .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PAPT: Periodic-Aware Transformer with Polynomial Trend Fitting for General Time Series Forecasting

  • Xiuyuan Wei,
  • Jiadong Chen,
  • Yuchen Fang,
  • Yinbo Sun,
  • Xiaofeng Gao,
  • Lintao Ma,
  • Guihai Chen

摘要

Time series forecasting has been widely applied across multiple domains including databases, finance and transportation. While Transformer-based models have made significant progress in capturing long-term dependencies through attention mechanisms, they struggle with modeling mixed multi-scale periodicity and dynamic trends. In this study, we propose PAPT, a Transformer-based general framework to capture multi-periodicity and long-term trends in time series. On one hand, we introduce Periodic Attention Positional Encoding (PAPE), which incorporates multiple periodic patterns into the attention encoding to enhance weight correlations at periodic synchronization points in the attention matrix. Furthermore, we employ a two-layer feature generation module (TGM) to capture fluctuations within cycles. On the other hand, we design an adaptive polynomial basis trend fitting module (PTM) that learns trend features through weight adjustment and fits them with multiple distinct polynomial bases. Compared with 8 baselines, PAPT demonstrates state-of-the-art forecasting performance through extensive experiments conducted on 10 real-world datasets in 4 domains. The code and datasets are shown in https://github.com/wxy1225/Periodic-Attention-with-Polynomial-Trend-Fitting-Transformer.git .