We investigate the robustness of the Radon machine, a general aggregation scheme for federated learning that distributes training by fitting and aggregating multiple base hypotheses. The Radon machine relies on iterated Radon point computations and promises to be robust to individual bad base hypotheses. However, we show that it is possible to exploit the unique geometrical properties of the Radon point to arbitrarily manipulate the output of the Radon machine. First, we show how to manipulate an individual Radon point. Then, we take into account the random aggregations performed by the Radon machine to provide a probabilistic bound for its worst-case outlier tolerance. Our experiments highlight the effect of additive noise during hypotheses aggregation.