Rate-adaptive RDMA congestion control for AI clusters
摘要
The rapid growth of AI models has imposed increasingly stringent performance demands on data centers. Modern data centers adopt Remote Direct Memory Access (RDMA) to reduce CPU overhead and network latency. RDMA operates over a lossless network, and RDMA congestion control (CC) protocols are key enablers for achieving low-latency and high-throughput data delivery. Through in-depth experiment analysis, we reveal that existing RDMA CC protocols still suffer from sluggish congestion response and convergence speed, limiting their ability to support latency-sensitive AI applications. In this paper, we propose a switch-driven CC algorithm named FACC. FACC enables switches to precisely identify flows that actually cause congestion and promptly notifies the congestion information to senders. At the sender, FACC leverages the intrinsic packet conservation property of a lossless network to assess the extent of network congestion. Then, FACC employs a PI controller to adaptively adjust the sending rate, thereby achieving rapid congestion elimination and improving both the transmission rate and convergence speed. We conduct extensive experiments to evaluate the performance of FACC. The results show that FACC improves convergence speed while achieving at most 86.6% lower flow completion time (FCT) compared with state-of-the-art approaches.