Multi-Agent Reinforcement Learning (MARL) enables multiple autonomous agents to simultaneously learn and make decisions in complex, interactive environments. Among various MARL paradigms, Independent Learning (IL) remains a dominant approach due to its simplicity and scalability, where each agent optimizes its policy independently, treating others as part of the environment. While IL eliminates the need for explicit inter-agent communication, existing MARL implementations fail to exploit its inherent parallelism. Current implementations train agents sequentially on a single accelerator, leading to severe underutilization of modern compute resources, particularly on multi-GPU platforms. In this work, we propose a multi-GPU training scheme that efficiently distributes independent agent policies across compute devices without altering the original IL semantics. To further enhance scalability, we design a dynamic load-balancing strategy that adaptively assigns training workloads based on computational demands and the varying capabilities of different GPUs, ensuring efficient utilization of hardware resources. Our approach achieves up to 15.5 \(\times \) higher throughput than state-of-the-art MARL implementations, demonstrating that fully leveraging the parallelism of IL can significantly accelerate MARL training, opening new possibilities for large-scale multi-agent learning in high-dimensional environments. We open-source our work with optimized implementations of widely used independent learning algorithms, enabling scalable MARL training on diverse accelerator platforms.