Federated learning (FL) allows multiple clients to train a model without sharing data. However, the decentralized nature of FL may make the model vulnerable to attacks such as model poisoning attacks (MPA), which aim to reduce the accuracy of the central model. Most current defense mechanisms perform significantly well against model poisoning attacks under the scenario where the attacker has access to a few genuine clients. However, if an attacker injects a sufficiently large number of fake clients into the FL system, this type of attack is referred to as a Model Poisoning Attack based on Fake Clients (MPAF), conventional defense methods may fail to recognize the attack and protect the global model. To address this challenge, we propose an effective defense mechanism against the MPAF that can accurately detect all fake clients and disregard their contribution to the aggregation process. Specifically, we analyze the sum of the final layer’s bias gradients and demonstrate that this sum theoretically should be zero for genuine classification models. The evaluation results show that, regardless of the number of fake clients, our proposed method could precisely detect all of them and maintain the system’s performance as it has not been attacked. Also, our proposed defense method exhibits much lower computational complexity while outperforming the others in system accuracy compared to the other defense methods.