Scholarlink

<p>Extracting universally applicable representations from unlabeled time series data is difficult but very beneficial in real-world scenarios. Current research predominantly adopts contrastive learning methods to address this challenge, often relying on data augmentation and sampling strategies for positive and negative pairs. However, this may lead to excessive inductive biases and introduce additional biases during data augmentation. Additionally, current studies often overlook frequency-domain features or fail to effectively integrate time- and frequency-domain information. Therefore, we propose a new framework (Time Series Self-Supervised Learning with Time–Frequency Masked Autoencoders) called TS-TFMAE. TS-TFMAE employs a more challenging cross-domain cross-reconstruction task to generate self-supervised signals to ensure the fusion of time–frequency information‌ for the first time. Specifically, we first employ time window segmentation instead of pointwise modeling and apply a high-coverage mask to create corrupted input data. We subsequently use self-attention to learn universal representations of visible regions. Afterward, we design an innovative time–frequency cross-attention module that reconstructs masked frequency and time-domain regions by leveraging visible representations in the time and frequency domains—a highly challenging pretext task. Finally, we introduce time–frequency consistency loss, which ensures consistency in time series representations. We carry out sufficient experiments on six public datasets, and the results demonstrate that TS-TFMAE significantly outperforms previous state-of-the-art baselines. Notably, TS-TFMAE achieves accuracy of 20.87% on PS, 91.32% on PTB-XL, and 96.47% on Epilepsy datasets, surpassing all compared methods. Moreover, it exhibits superior robustness under noisy conditions, with only a minimal performance drop at 0 dB SNR.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

TS-TFMAE: time series self-supervised learning with time–frequency masked autoencoders

  • 期刊论文

  • 发表日期：

    2026年3月16日

查看全文
  • Songbai Liu,
  • Hongru Li,
  • Yinghua Yang

摘要

Extracting universally applicable representations from unlabeled time series data is difficult but very beneficial in real-world scenarios. Current research predominantly adopts contrastive learning methods to address this challenge, often relying on data augmentation and sampling strategies for positive and negative pairs. However, this may lead to excessive inductive biases and introduce additional biases during data augmentation. Additionally, current studies often overlook frequency-domain features or fail to effectively integrate time- and frequency-domain information. Therefore, we propose a new framework (Time Series Self-Supervised Learning with Time–Frequency Masked Autoencoders) called TS-TFMAE. TS-TFMAE employs a more challenging cross-domain cross-reconstruction task to generate self-supervised signals to ensure the fusion of time–frequency information‌ for the first time. Specifically, we first employ time window segmentation instead of pointwise modeling and apply a high-coverage mask to create corrupted input data. We subsequently use self-attention to learn universal representations of visible regions. Afterward, we design an innovative time–frequency cross-attention module that reconstructs masked frequency and time-domain regions by leveraging visible representations in the time and frequency domains—a highly challenging pretext task. Finally, we introduce time–frequency consistency loss, which ensures consistency in time series representations. We carry out sufficient experiments on six public datasets, and the results demonstrate that TS-TFMAE significantly outperforms previous state-of-the-art baselines. Notably, TS-TFMAE achieves accuracy of 20.87% on PS, 91.32% on PTB-XL, and 96.47% on Epilepsy datasets, surpassing all compared methods. Moreover, it exhibits superior robustness under noisy conditions, with only a minimal performance drop at 0 dB SNR.