Scholarlink

<p>Climate change makes accurate weather prediction and large-scale data analysis increasingly crucial, but the sheer volume of weather data strains data storage and sharing. Here we introduce Aeolus, a deep learning framework with powerful Variational Auto-Encoder transFormer (VAEFormer) modules that dramatically reduces the size of weather datasets, compressing the widely used 400-terabyte reanalysis atmospheric dataset to just 0.85 TeraBytes (TB)-a compression ratio of 470 &#xa0;×&#xa0;. Aeolus surpasses traditional image compression methods such as Joint Photographic Experts Group 2000 (JPEG2000), which typically achieve much lower compression rates on weather data. It also enables fast data processing speeds of over 1 GigaByte (GB) per second, saving computational resources. Our tests show Aeolus maintains high accuracy, with temperature errors as low as 0.17<sup>∘</sup> Kelvin and reliable climate patterns. These advances make it easier for researchers to access and analyze massive weather datasets, supporting better weather forecasting and climate studies. Aeolus thus offers a valuable tool for climate research and efforts to address global environmental challenges.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Climate science data can be compressed efficiently by dual-stage extreme compression with a variational auto-encoder transformer

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月24日

查看全文
  • Tao Han,
  • Zhenghao Chen,
  • Song Guo,
  • Wanghan Xu,
  • Wanli Ouyang,
  • Lei Bai

摘要

Climate change makes accurate weather prediction and large-scale data analysis increasingly crucial, but the sheer volume of weather data strains data storage and sharing. Here we introduce Aeolus, a deep learning framework with powerful Variational Auto-Encoder transFormer (VAEFormer) modules that dramatically reduces the size of weather datasets, compressing the widely used 400-terabyte reanalysis atmospheric dataset to just 0.85 TeraBytes (TB)-a compression ratio of 470  × . Aeolus surpasses traditional image compression methods such as Joint Photographic Experts Group 2000 (JPEG2000), which typically achieve much lower compression rates on weather data. It also enables fast data processing speeds of over 1 GigaByte (GB) per second, saving computational resources. Our tests show Aeolus maintains high accuracy, with temperature errors as low as 0.17∘ Kelvin and reliable climate patterns. These advances make it easier for researchers to access and analyze massive weather datasets, supporting better weather forecasting and climate studies. Aeolus thus offers a valuable tool for climate research and efforts to address global environmental challenges.