Scholarlink

<p>The integration of machine learning (ML) into modern data management systems has enabled intelligent decision-making across large-scale information infrastructures. However, existing performance benchmarks typically evaluate isolated components such as model training or inference, leaving the behavior of full end-to-end pipelines poorly understood. This paper introduces PipeBench, a reproducible benchmarking framework for evaluating integrated ML pipelines that combine distributed data processing, automated machine learning (AutoML), model management, and production serving. The proposed framework is implemented using widely adopted open-source technologies including Apache Spark, Kubeflow, MLflow, and TensorFlow Serving. Experiments are conducted across three representative datasets (industrial IoT, e-commerce, and financial analytics) under controlled variations of data scale, cluster size, and AutoML strategies. Results show that model training dominates pipeline latency (58–70%), while preprocessing contributes 19–28% of execution time. Bayesian optimization achieves 97% of the accuracy of evolutionary AutoML methods while reducing computational cost by nearly half. A multi-fidelity approach (BOHB) further improves efficiency, achieving equivalent accuracy at 43% lower cost than standard Bayesian optimization. The study also identifies key system bottlenecks including GPU underutilization, shuffle-induced memory pressure, and storage contention during checkpointing. All code, configuration files, and synthetic data generators are publicly released to ensure reproducibility. PipeBench provides a standardized experimental framework and practical design guidelines for building scalable ML-enabled information systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

PipeBench: a benchmarking framework for end-to-end machine learning pipelines

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月28日

查看全文
  • Sanjay Agal,
  • Ruchika Katariya

摘要

The integration of machine learning (ML) into modern data management systems has enabled intelligent decision-making across large-scale information infrastructures. However, existing performance benchmarks typically evaluate isolated components such as model training or inference, leaving the behavior of full end-to-end pipelines poorly understood. This paper introduces PipeBench, a reproducible benchmarking framework for evaluating integrated ML pipelines that combine distributed data processing, automated machine learning (AutoML), model management, and production serving. The proposed framework is implemented using widely adopted open-source technologies including Apache Spark, Kubeflow, MLflow, and TensorFlow Serving. Experiments are conducted across three representative datasets (industrial IoT, e-commerce, and financial analytics) under controlled variations of data scale, cluster size, and AutoML strategies. Results show that model training dominates pipeline latency (58–70%), while preprocessing contributes 19–28% of execution time. Bayesian optimization achieves 97% of the accuracy of evolutionary AutoML methods while reducing computational cost by nearly half. A multi-fidelity approach (BOHB) further improves efficiency, achieving equivalent accuracy at 43% lower cost than standard Bayesian optimization. The study also identifies key system bottlenecks including GPU underutilization, shuffle-induced memory pressure, and storage contention during checkpointing. All code, configuration files, and synthetic data generators are publicly released to ensure reproducibility. PipeBench provides a standardized experimental framework and practical design guidelines for building scalable ML-enabled information systems.