Scholarlink

<p>Internet of Things (IoT) applications often require real-time processing of data streams generated by a rapidly growing number of geographically distributed smart devices. Traditional cloud-centric processing introduces significant end-to-end latency due to the transmission of data to remote cloud data centers. Cloud–edge infrastructures have emerged as a promising solution by extending computational resources closer to data sources. However, efficient operator placement for Data Stream Processing (DSP) applications within hybrid cloud–edge environments remains a challenging problem. While recent researches have primarily focused on single-application placement or optimize a limited set of objectives in operator placement across heterogeneous cloud–edge devices, two major aspects remain unaddressed: (1) handling trade-offs between latency, network usage, application owner’s preferences, QoS constraints, and heterogeneous resource utilization and (2) managing concurrent execution of multiple DSP applications, rather than treating them sequentially. This paper proposes a resource-aware multi-application operator placement method based on Deep Reinforcement Learning (DRL) for heterogeneous cloud–edge environments. The proposed approach jointly optimizes end-to-end latency and network usage while satisfying QoS requirements, application owners’ preferences, and resource constraints. Unlike existing methods that process applications sequentially, the proposed scheduler considers multiple DSP applications simultaneously, enabling more efficient utilization of distributed resources and reducing the impact of application submission order. A Deep Q-Network (DQN) agent is employed to learn near-optimal placement. Furthermore, a proof-of-concept prototype is implemented using Apache Storm to validate the proposed approach in a realistic deployment setting. Experimental results demonstrate that the proposed method achieves a favorable balance between response time and network usage across varying numbers of applications, resource providers, and workload configurations. Comparative evaluations against traditional operator placement methods and a DRL-based method show that the proposed DQN-based scheduler consistently delivers lower response time, reduced network consumption, faster convergence, and improved scalability in heterogeneous cloud–edge environments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-application operator placement in cloud–edge environments: a Deep Reinforcement Learning approach for big Data Stream Processing

  • 期刊论文

  • 发表日期：

    2026年7月21日

查看全文
  • Simin Ghasemi-Falavarjani,
  • Behrouz Shahgholi Ghahfarokhi,
  • Mohammadali Nematbakhsh,
  • Nikolaos Georgantas

摘要

Internet of Things (IoT) applications often require real-time processing of data streams generated by a rapidly growing number of geographically distributed smart devices. Traditional cloud-centric processing introduces significant end-to-end latency due to the transmission of data to remote cloud data centers. Cloud–edge infrastructures have emerged as a promising solution by extending computational resources closer to data sources. However, efficient operator placement for Data Stream Processing (DSP) applications within hybrid cloud–edge environments remains a challenging problem. While recent researches have primarily focused on single-application placement or optimize a limited set of objectives in operator placement across heterogeneous cloud–edge devices, two major aspects remain unaddressed: (1) handling trade-offs between latency, network usage, application owner’s preferences, QoS constraints, and heterogeneous resource utilization and (2) managing concurrent execution of multiple DSP applications, rather than treating them sequentially. This paper proposes a resource-aware multi-application operator placement method based on Deep Reinforcement Learning (DRL) for heterogeneous cloud–edge environments. The proposed approach jointly optimizes end-to-end latency and network usage while satisfying QoS requirements, application owners’ preferences, and resource constraints. Unlike existing methods that process applications sequentially, the proposed scheduler considers multiple DSP applications simultaneously, enabling more efficient utilization of distributed resources and reducing the impact of application submission order. A Deep Q-Network (DQN) agent is employed to learn near-optimal placement. Furthermore, a proof-of-concept prototype is implemented using Apache Storm to validate the proposed approach in a realistic deployment setting. Experimental results demonstrate that the proposed method achieves a favorable balance between response time and network usage across varying numbers of applications, resource providers, and workload configurations. Comparative evaluations against traditional operator placement methods and a DRL-based method show that the proposed DQN-based scheduler consistently delivers lower response time, reduced network consumption, faster convergence, and improved scalability in heterogeneous cloud–edge environments.