Task Scheduling in Multi-layer CLOS Topology Networks Based on Shortest Path
摘要
With the rapid development of AI, the demand for computing resources in model training is increasing. This study optimizes the scheduling efficiency and traffic management of training tasks under multi-layer CLOS topology network, and constructs a model considering the specific connection, bandwidth limitation, and traffic conflict between NPU and three types of switches. The research abstracts the whole system into an undirected graph model, uses an improved circle avoidance method to minimize the average path length, and uses Dijkstra and Floyd-Warshall algorithms to calculate the shortest path between nodes. Finally, this paper conducts a sensitivity analysis to discuss how the number of network recursive layers or two types of switch ports influences the shortest length, which evaluates the model’s performance under various parameter changes, thereby verifying the scalability and robustness of the model.