Federated learning (FL) is a distributed machine learning (ML) scheme that enables collaborative model training without compromising data privacy. In the TinyML community, communication is often prioritized over computation due to the limited bandwidth of the client devices. This paper aims to explore the rationale behind this perception by focusing on small models with approximately 1M parameters and investigating the effects of quantization and sparsification techniques for computation and communication purposes. We introduce mathematical models essential for estimating the time and energy consumption of computation and communication in an FL setup. Our research explores various configurations that impact FL efficiency, including model quantization and sparsification, processor frequency, batch sizes, and ML architecture. Notably, we demonstrate that for small models, communication energy can be negligible compared to computational energy. Consequently, contrary to existing literature, we propose that sparsification for communication may not be necessary as it introduces additional computational effort and energy consumption to achieve a target accuracy without significant gains in communication energy. This research enhances our understanding of the trade-offs between communication and computation in FL, particularly for small models like in TinyML. By emphasizing the importance of computational energy, our findings provide valuable insights for the design and optimization of FL systems.