Federated Learning (FL) can help train collaborative machine learning models while preserving individual client privacy by keeping data local and private. Among other challenges, data heterogeneity brings extensive hurdles in learning collaborative models, as clients usually own non-independent and identically distributed (non-IID) data. One solution to improve model performance under non-IID data is implicit data augmentation, which requires global distribution modeling. This work proposes FedMGAN, a two-phase novel multi-tier generative adversarial network (GAN) based global data distribution modeling approach to tackle data heterogeneity in FL. In the first phase, FedMGAN utilizes a subset of clients for GAN training, whose generators are further improved against the remaining clients’ discriminators. In the second phase, the trained GAN is used to improve the performance of the global FL model. FedMGAN performs better than the current state-of-the-art FL approaches on four different benchmarking datasets under non-IID data distribution.