Federated Learning (FL) is a significant distributed learning setting in large scale machine learning. In FL, agents with access to their own data combine models learned from local data to build a global mode. If the agents’ data are distributed differently, then FL might give a biased global model which may not be optimal for each agent. This causes a question for the agents: should they join the federation? In this work, we assume that the agents have different amounts of data and different data distributions, and there are noises when they draw the data. Each agent’s goal is to minimize the expected variance squared error (VSE) on its own distribution. Three different federating models are analyzed: In uniform federation, all agents use a single model; In coarse-grained federation, each agent has a parameter to weight the global model with its own local model; In fine-grained federation, each agent uses a vector of weights to combine other agents’ contribution. We analyze the scenario where agents may divide into coalitions where the agents in the same coalition jointly construct models under the framework of hedonic game theory. The stable partitions are thoroughly studied for uniform federation model.