Scholarlink

<p>The wave of digital transformation, characterized by a massive increase in data production by humans and machines, is driving increased demand for resources and putting more pressure on Big Data platform management systems. Revolutionary resource allocation strategies are therefore required to meet demand without compromising performance. This need is particularly acute in heterogeneous computing environments, where devices of different generations and performance levels operate together. This heterogeneity across nodes represents an additional constraint that must not be overlooked when allocating resources to avoid any impact on system performance. To address these challenges, this paper proposes a Genetic Algorithm–Based Scheduler (GABS) for heterogeneous Hadoop clusters. The scheduler considers job priority, node heterogeneity, data locality, and overall execution cost in task assignment to optimize resource utilization and enhance overall system performance. Experimental evaluations using standard Hadoop benchmarks demonstrate that GABS reduces processing time by up to 27%, increases system throughput by 35%, decreases execution cost by 29%, and enhances data locality by 24% compared to Hadoop’s default schedulers. These results highlight the effectiveness of GABS in enhancing scalability, performance, and resource efficiency in heterogeneous computing environments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

GA-Based Scheduling Algorithm for Heterogeneous Hadoop Clusters

  • 期刊论文

  • 发表日期：

    2026年7月24日

查看全文
  • Nour-eddine Bakni,
  • Ismail Assayad

摘要

The wave of digital transformation, characterized by a massive increase in data production by humans and machines, is driving increased demand for resources and putting more pressure on Big Data platform management systems. Revolutionary resource allocation strategies are therefore required to meet demand without compromising performance. This need is particularly acute in heterogeneous computing environments, where devices of different generations and performance levels operate together. This heterogeneity across nodes represents an additional constraint that must not be overlooked when allocating resources to avoid any impact on system performance. To address these challenges, this paper proposes a Genetic Algorithm–Based Scheduler (GABS) for heterogeneous Hadoop clusters. The scheduler considers job priority, node heterogeneity, data locality, and overall execution cost in task assignment to optimize resource utilization and enhance overall system performance. Experimental evaluations using standard Hadoop benchmarks demonstrate that GABS reduces processing time by up to 27%, increases system throughput by 35%, decreases execution cost by 29%, and enhances data locality by 24% compared to Hadoop’s default schedulers. These results highlight the effectiveness of GABS in enhancing scalability, performance, and resource efficiency in heterogeneous computing environments.