Scholarlink

<p>Memory efficiency is a critical bottleneck in deep learning models for sequence processing, particularly in long-range dependencies and continuous data streams. In the effort to solve this, we introduce a new architecture denoted as Reversible Temporal Transformer (TempVerseFormer). TempVerseFormer integrates reversible transformer blocks uniquely with a time-agnostic backpropagation strategy that decouples the memory footprint from the temporal depth and enables efficient training on long prediction time ranges. We validate our model on both a procedurally generated dataset designed to probe specific temporal dynamics and on standard spatio-temporal benchmarks, including KTH Action and TaxiBJ. The predictive accuracy of TempVerseFormer is competitive compared to other tested baselines, with memory consumption being practically independent of the time-to-predict. This substantial gain in memory efficiency, achieved while maintaining strong performance on synthetic and real-world benchmarks, places the TempVerseFormer as an indicative candidate for scalable temporal sequence modeling, allowing for real-time adaptation or video analysis in resource-constrained environments, and thereby leading to more fiscally responsible and temporally wise resource AI systems that are capable of working with changing and evolving environments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Temporal modeling with reversible transformers

  • 期刊论文

  • 发表日期：

    2026年2月3日

查看全文
  • Leonid Kulyk

摘要

Memory efficiency is a critical bottleneck in deep learning models for sequence processing, particularly in long-range dependencies and continuous data streams. In the effort to solve this, we introduce a new architecture denoted as Reversible Temporal Transformer (TempVerseFormer). TempVerseFormer integrates reversible transformer blocks uniquely with a time-agnostic backpropagation strategy that decouples the memory footprint from the temporal depth and enables efficient training on long prediction time ranges. We validate our model on both a procedurally generated dataset designed to probe specific temporal dynamics and on standard spatio-temporal benchmarks, including KTH Action and TaxiBJ. The predictive accuracy of TempVerseFormer is competitive compared to other tested baselines, with memory consumption being practically independent of the time-to-predict. This substantial gain in memory efficiency, achieved while maintaining strong performance on synthetic and real-world benchmarks, places the TempVerseFormer as an indicative candidate for scalable temporal sequence modeling, allowing for real-time adaptation or video analysis in resource-constrained environments, and thereby leading to more fiscally responsible and temporally wise resource AI systems that are capable of working with changing and evolving environments.