Robot faults are inevitable in swarm systems deployed in real-world applications, yet most fault mitigation approaches rely on explicit fault detection pipelines and hand-crafted recovery responses. We propose a novel Multi-Agent Reinforcement Learning (MARL) approach that learns a fault-robust controller by reactively mapping local state metrics directly to a set of predefined mitigation actions. We train a shared-parameter, recurrent MARL policy using Centralised Training with Decentralised Execution (CTDE) in a box transport task. In our approach, the same policy is used by both faulty and non-faulty robots, and in different fault conditions. Our learned policy demonstrates robustness to faults compared to a non-learning baseline, and scales to increasing numbers of faults not encountered during training. Results demonstrate that MARL provides a practical approach to robustness against faults in robot swarms, without relying on explicit fault detection.