Scholarlink

<p>As the Internet of Things (IoT) scales, the Computing Continuum relies increasingly on distributed stream processing platforms like Apache Kafka to bridge edge data generation with cloud analytics. However, in unstable edge environments, where high packet loss and jitter make throughput-centric tuning detrimental to latency and energy efficiency, standard Kafka configurations and current linear-scaling heuristics fail. In order to overcome this, we present Net-Kafka, a network-contextualized auto-tuning framework that uses Online Bayesian Optimization instead of strict heuristic rules. Unlike prior approaches that optimize solely for message rate, Net-Kafka models the configuration landscape as a non-linear Gaussian Process, explicitly incorporating network telemetry (Round Trip Time, Packet Loss) into the decision state space. This makes it possible for the agent to independently negotiate the intricate trade-offs between device power consumption, retransmission overhead, and batching efficiency. Comprehensive tests on a real testbed with Raspberry Pi gateways and simulated 4&#xa0;G/LoRa links show that Net-Kafka sustains 95% throughput stability under 5% packet loss–conditions where the most advanced LSTM techniques fail. Furthermore, the system achieves a 39% reduction in consumer lag and a 22% improvement in energy efficiency compared to baseline methods, confirming the necessity of network-aware probabilistic optimization in the edge-cloud continuum.The source code for Net-Kafka is available at <a href="https://github.com/ahmadpanah/Net-Kafka">https://github.com/ahmadpanah/Net-Kafka</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Net-Kafka: network-contextualized Bayesian optimization for resilient stream processing in lossy IoT environments

  • 期刊论文

  • 发表日期：

    2026年5月4日

查看全文
  • Seyed Hossein Ahmadpanah

摘要

As the Internet of Things (IoT) scales, the Computing Continuum relies increasingly on distributed stream processing platforms like Apache Kafka to bridge edge data generation with cloud analytics. However, in unstable edge environments, where high packet loss and jitter make throughput-centric tuning detrimental to latency and energy efficiency, standard Kafka configurations and current linear-scaling heuristics fail. In order to overcome this, we present Net-Kafka, a network-contextualized auto-tuning framework that uses Online Bayesian Optimization instead of strict heuristic rules. Unlike prior approaches that optimize solely for message rate, Net-Kafka models the configuration landscape as a non-linear Gaussian Process, explicitly incorporating network telemetry (Round Trip Time, Packet Loss) into the decision state space. This makes it possible for the agent to independently negotiate the intricate trade-offs between device power consumption, retransmission overhead, and batching efficiency. Comprehensive tests on a real testbed with Raspberry Pi gateways and simulated 4 G/LoRa links show that Net-Kafka sustains 95% throughput stability under 5% packet loss–conditions where the most advanced LSTM techniques fail. Furthermore, the system achieves a 39% reduction in consumer lag and a 22% improvement in energy efficiency compared to baseline methods, confirming the necessity of network-aware probabilistic optimization in the edge-cloud continuum.The source code for Net-Kafka is available at https://github.com/ahmadpanah/Net-Kafka