Scholarlink

<p>Task failures in large-scale cloud clusters degrade reliability and waste resources, yet failure prediction studies often rely on subsampled traces because end-to-end processing at full scale is challenging. We present an at-scale, reproducible pipeline for task-failure prediction using the Google Cluster Trace 2019 by processing the complete “Instance events” records across all eight clusters (≈ 163 GB, 14,127 parquet files) into a compact, machine-learning-ready feature table. We formulate failure prediction as a regression problem over aggregated trace windows and evaluate regularized linear baselines (Ridge, LASSO), Extreme Learning Machine (ELM), and ELM hybrids tuned with nature-inspired optimizers such as Genetic Algorithm, Grey Wolf Optimizer, and Swarm-based simulated annealing. Models are assessed with time-ordered train/validation/test splits and repeated runs with mean ± SD, <i>N</i> = 5, to reduce temporal leakage and quantify variability. The best-performing model, ELM_SwarmSA, achieves R² = 0.9913 ± 0.0004 with MAE = 3,054 ± 31 and RMSE = 4,314 ± 40, outperforming simpler baselines. By enabling full-trace benchmarking and providing processing artifacts and a compact derived dataset, this work improves the reliability of empirical comparisons and supports practical failure-risk scoring to inform scheduling and resource management.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Hybrid ML models for task-failure prediction on google cluster traces

  • 期刊论文

  • 发表日期：

    2026年7月20日

查看全文
  • Danyal Shahmirzadi,
  • Wen-Fong Wang,
  • Amir Masoud Rahmani

摘要

Task failures in large-scale cloud clusters degrade reliability and waste resources, yet failure prediction studies often rely on subsampled traces because end-to-end processing at full scale is challenging. We present an at-scale, reproducible pipeline for task-failure prediction using the Google Cluster Trace 2019 by processing the complete “Instance events” records across all eight clusters (≈ 163 GB, 14,127 parquet files) into a compact, machine-learning-ready feature table. We formulate failure prediction as a regression problem over aggregated trace windows and evaluate regularized linear baselines (Ridge, LASSO), Extreme Learning Machine (ELM), and ELM hybrids tuned with nature-inspired optimizers such as Genetic Algorithm, Grey Wolf Optimizer, and Swarm-based simulated annealing. Models are assessed with time-ordered train/validation/test splits and repeated runs with mean ± SD, N = 5, to reduce temporal leakage and quantify variability. The best-performing model, ELM_SwarmSA, achieves R² = 0.9913 ± 0.0004 with MAE = 3,054 ± 31 and RMSE = 4,314 ± 40, outperforming simpler baselines. By enabling full-trace benchmarking and providing processing artifacts and a compact derived dataset, this work improves the reliability of empirical comparisons and supports practical failure-risk scoring to inform scheduling and resource management.