This chapter addresses computational acceleration techniques for scalable GNN training. At the model level, prevalent frameworks incur significant overhead from batch preparation and CPU-GPU data transfer in mini-batch training. Furthermore, their constrained device utilization patterns restrict opportunities for pipeline parallelism. To overcome these limitations, we propose DAHA, a GNN training framework featuring data-aware and hardware-aware execution planning that optimizes end-to-end training workflows. Extensive experiments confirm that DAHA delivers consistent and substantial speedups while generalizing across diverse message-passing GNN architectures. At the operator level, prior approaches for accelerating sparse operators often decompose sparse tensors and search for hybrid format combinations tailored to specific sparsity patterns. However, these methods face a fundamental trade-off between search space size and exploration time, which can limit achievable efficiency. To resolve this, we propose STile, a framework that expands the optimization space both in breadth through flexible sparse tensor transformations and in depth via multi-level decomposition. Empirical evaluations of SpMM and SDDMM operators across varied sparsity patterns demonstrate the effectiveness of STile.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Accelerating Computation for Scalable Graph Neural Network Training

  • Jingzhi Fang,
  • Zhiyuan Li

摘要

This chapter addresses computational acceleration techniques for scalable GNN training. At the model level, prevalent frameworks incur significant overhead from batch preparation and CPU-GPU data transfer in mini-batch training. Furthermore, their constrained device utilization patterns restrict opportunities for pipeline parallelism. To overcome these limitations, we propose DAHA, a GNN training framework featuring data-aware and hardware-aware execution planning that optimizes end-to-end training workflows. Extensive experiments confirm that DAHA delivers consistent and substantial speedups while generalizing across diverse message-passing GNN architectures. At the operator level, prior approaches for accelerating sparse operators often decompose sparse tensors and search for hybrid format combinations tailored to specific sparsity patterns. However, these methods face a fundamental trade-off between search space size and exploration time, which can limit achievable efficiency. To resolve this, we propose STile, a framework that expands the optimization space both in breadth through flexible sparse tensor transformations and in depth via multi-level decomposition. Empirical evaluations of SpMM and SDDMM operators across varied sparsity patterns demonstrate the effectiveness of STile.