Following an environmentally constrained path in a timely fashion can be crucial for swarms in scenarios such as disaster response or emergency evacuation. In such situations, swarms must rapidly follow potentially challenging paths while not losing cohesion. We benchmark a robust, decentralized gradient-following behavior against varying path sinuosity, and swarm size. The swarm races to reach a minimum distance in a fixed time budget. We measure the success rate and the mean completion time. Our findings show that the algorithm allows the swarm to successfully reach the target line in the allotted time as long as the swarm size is relatively small. Furthermore, we find that the algorithm handles low and medium degrees of sinuosity well but struggles with high sinuosity, where the turns become too sharp. This work is the first to study swarm racing in constrained environments, revealing that “more is not always better”: larger swarms hinder rapid path traversal. This paves the way for future research on scale-invariant racing collective behaviors.