With the rapid development of AI, the demand for computing resources in model training is increasing. This study optimizes the scheduling efficiency and traffic management of training tasks under multi-layer CLOS topology network, and constructs a model considering the specific connection, bandwidth limitation, and traffic conflict between NPU and three types of switches. The research abstracts the whole system into an undirected graph model, uses an improved circle avoidance method to minimize the average path length, and uses Dijkstra and Floyd-Warshall algorithms to calculate the shortest path between nodes. Finally, this paper conducts a sensitivity analysis to discuss how the number of network recursive layers or two types of switch ports influences the shortest length, which evaluates the model’s performance under various parameter changes, thereby verifying the scalability and robustness of the model.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Task Scheduling in Multi-layer CLOS Topology Networks Based on Shortest Path

  • Xuan Du,
  • Jianan Chen,
  • Wei Xie

摘要

With the rapid development of AI, the demand for computing resources in model training is increasing. This study optimizes the scheduling efficiency and traffic management of training tasks under multi-layer CLOS topology network, and constructs a model considering the specific connection, bandwidth limitation, and traffic conflict between NPU and three types of switches. The research abstracts the whole system into an undirected graph model, uses an improved circle avoidance method to minimize the average path length, and uses Dijkstra and Floyd-Warshall algorithms to calculate the shortest path between nodes. Finally, this paper conducts a sensitivity analysis to discuss how the number of network recursive layers or two types of switch ports influences the shortest length, which evaluates the model’s performance under various parameter changes, thereby verifying the scalability and robustness of the model.