Multivariate time series data generated in fields such as transportation, economics, and healthcare can support various analytical tasks. However, due to the common occurrence of missing data in real-world scenarios, existing work faces significant challenges in ensuring data interpretability and effectively conducting specific tasks like advanced analysis and pattern recognition. Traditional missing value imputation methods struggle to effectively capture temporal dependencies in time series data. Although deep learning-based approaches (e.g., Transformer) demonstrate advantages in modeling long-range dependencies, they generally suffer from two critical limitations: (1) inadequate consideration of unequal time intervals between adjacent observations; (2) deficiencies in joint modeling of temporal and feature dependencies. To address these gaps, this paper proposes an attention-based framework named TimeMultiformer. The framework innovatively integrates Transformer and iTransformer architectures to accurately capture temporal dependencies and inter-variable feature dependencies, respectively. Specifically, it introduces a time lag matrix to explicitly model the impact of historical observations on current imputation and employs a diagonal-masked multi-head attention mechanism to mitigate the model’s over-reliance on current time-step information, thereby enhancing the capture of long-range dependencies. Additionally, through a joint data imputation learning strategy, TimeMultiformer exhibits superior performance in time series imputation tasks. Extensive experiments on four real-world datasets demonstrate that TimeMultiformer outperforms state-of-the-art methods in time series imputation both quantitatively and qualitatively, with the mean absolute error (MAE) reduced by 2%–16%, root mean square error (RMSE) reduced by 10%–26%, and the average reduction of mean relative error (MRE) exceeding 10%. The source code of this project is publicly available at: https://github.com/wzq-come-on/TimeMultiformer.git .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TimeMultiformer: Attention-Based Collaborative Feature and Temporal Dependencies Learning for Multivariate Time Series Imputation

  • Zhaoqin Wang,
  • Qian Ma,
  • Hui Li,
  • Qiao Ning,
  • Furui Zhan,
  • Shikai Guo

摘要

Multivariate time series data generated in fields such as transportation, economics, and healthcare can support various analytical tasks. However, due to the common occurrence of missing data in real-world scenarios, existing work faces significant challenges in ensuring data interpretability and effectively conducting specific tasks like advanced analysis and pattern recognition. Traditional missing value imputation methods struggle to effectively capture temporal dependencies in time series data. Although deep learning-based approaches (e.g., Transformer) demonstrate advantages in modeling long-range dependencies, they generally suffer from two critical limitations: (1) inadequate consideration of unequal time intervals between adjacent observations; (2) deficiencies in joint modeling of temporal and feature dependencies. To address these gaps, this paper proposes an attention-based framework named TimeMultiformer. The framework innovatively integrates Transformer and iTransformer architectures to accurately capture temporal dependencies and inter-variable feature dependencies, respectively. Specifically, it introduces a time lag matrix to explicitly model the impact of historical observations on current imputation and employs a diagonal-masked multi-head attention mechanism to mitigate the model’s over-reliance on current time-step information, thereby enhancing the capture of long-range dependencies. Additionally, through a joint data imputation learning strategy, TimeMultiformer exhibits superior performance in time series imputation tasks. Extensive experiments on four real-world datasets demonstrate that TimeMultiformer outperforms state-of-the-art methods in time series imputation both quantitatively and qualitatively, with the mean absolute error (MAE) reduced by 2%–16%, root mean square error (RMSE) reduced by 10%–26%, and the average reduction of mean relative error (MRE) exceeding 10%. The source code of this project is publicly available at: https://github.com/wzq-come-on/TimeMultiformer.git .