Rapidly expanding Electric Vehicle (EV) adoption necessitates robust, large-scale charging strategies to meet global decarbonization targets. Traditional methods, such as heuristics and mathematical programming-based approaches, struggle to scale effectively and adapt to EV dispatch’s complexity, uncertainty, and variability. Reinforcement Learning (RL) offers a promising alternative due to its ability to handle complex optimization problems, process substantial real-time data, and learn continuously without explicit retraining for every scenario. This study proposes a novel end-to-end RL framework that leverages the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, enhanced by Graph Neural Networks (GNN), to capture spatial interactions among charging nodes. A key contribution is integrating a safety layer designed to ensure grid stability, preserve EV charging requirements, and enforce power limits. The RL agent was trained and evaluated using real EV charging sessions, offering a realistic assessment of its performance. The results indicate that the proposed method can efficiently coordinate large fleets of EVs, ensuring stable power grid operation and fair distribution of charging resources.