<p>The rapid growth of camera-based IoT devices demands the need for efficient video compression, particularly for edge applications where devices face hardware constraints, often with only 1 or 2 MB of RAM and unstable internet connections. Traditional and deep video compression methods are designed for high-end hardware, exceeding the capabilities of these constrained devices. Consequently, video compression in these scenarios is often limited to M-JPEG due to its high hardware efficiency and low5 complexity. This paper introduces <Emphasis FontCategory="NonProportional">MCUCoder</Emphasis>, an open-source adaptive bitrate video compression model tailored for resource-limited IoT settings. <Emphasis FontCategory="NonProportional">MCUCoder</Emphasis> features an ultra-lightweight encoder with only 10.5K parameters and a minimal 350KB memory footprint, making it well-suited for edge devices and MCUs. While <Emphasis FontCategory="NonProportional">MCUCoder</Emphasis> uses a similar amount of energy as M-JPEG, it reduces bitrate by 55.65% on the MCL-JCV dataset and 55.59% on the UVG dataset, measured in MS-SSIM. <Emphasis FontCategory="NonProportional">MCUCoder</Emphasis> supports adaptive-bitrate transmission by generating a latent representation sorted by importance, allowing the transmitted data to be adjusted according to available bandwidth. We further show that <Emphasis FontCategory="NonProportional">MCUCoder</Emphasis> compression outperforms M-JPEG compression when evaluated on downstream AI tasks, including image classification, object detection, and image captioning. Source code available at <a href="https://github.com/ds-kiel/MCUCoder">https://github.com/ds-kiel/MCUCoder</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Ultra-Lightweight Adaptive Bitrate Deep Video Compression

  • Ali Hojjat,
  • Janek Haberer,
  • Olaf Landsiedel

摘要

The rapid growth of camera-based IoT devices demands the need for efficient video compression, particularly for edge applications where devices face hardware constraints, often with only 1 or 2 MB of RAM and unstable internet connections. Traditional and deep video compression methods are designed for high-end hardware, exceeding the capabilities of these constrained devices. Consequently, video compression in these scenarios is often limited to M-JPEG due to its high hardware efficiency and low5 complexity. This paper introduces MCUCoder, an open-source adaptive bitrate video compression model tailored for resource-limited IoT settings. MCUCoder features an ultra-lightweight encoder with only 10.5K parameters and a minimal 350KB memory footprint, making it well-suited for edge devices and MCUs. While MCUCoder uses a similar amount of energy as M-JPEG, it reduces bitrate by 55.65% on the MCL-JCV dataset and 55.59% on the UVG dataset, measured in MS-SSIM. MCUCoder supports adaptive-bitrate transmission by generating a latent representation sorted by importance, allowing the transmitted data to be adjusted according to available bandwidth. We further show that MCUCoder compression outperforms M-JPEG compression when evaluated on downstream AI tasks, including image classification, object detection, and image captioning. Source code available at https://github.com/ds-kiel/MCUCoder.