Federated Learning has emerged as a promising approach to train machine learning models while preserving data privacy. However, its decentralized nature introduces unique vulnerabilities, particularly in non-Independent and Identically Distributed (non-IID) scenarios that reflect real-world conditions. This paper investigates model poisoning attacks in non-IID federated learning environments, focusing on Byzantine attacks where malicious participants send fake updates to compromise the global model behaviour. We implemented a realistic federated learning setup using the MNIST dataset, where participants had access to different classes, thus simulating the data heterogeneity typical of real application domains. The experiments assessed attacks targeting different neural network components—including all layers, convolutional layers, dense layers, and output layers. Our results show that even a single malicious participant can significantly degrade model performance. These findings reveal critical architectural vulnerabilities and emphasize the need for defense mechanisms in federated learning systems.