The continuous generation of massive amounts of data from connected smart objects, autonomous systems, and embedded systems, in real or delayed time, requires significant storage resources, as well as powerful computing resources and high-performance control and security systems. In this article, we will discuss the various research projects conducted by researchers with the aim of developing distributed, decentralized, and secure federated learning architectures. We begin with cloud-based FL architecture and an edge-based FL architecture, both of which allow IoT devices to leverage storage and computing resources for the reliable and efficient processing of decentralized learning models. Next, we introduce federated learning (FL) in an AI-based cloud architecture, which involves the use of an intelligent data transfer system to improve computation time and reduce latency between the cloud server and IoT devices. Then we introduce a modularized federated learning architecture for privacy and security considerations, which aims to ensure data security and confidentiality and create a privacy-enhancing process for data exchange between IoT devices. Finally, we introduce an architecture that integrates the learning model into a blockchain system while applying a POW (proof of work) consensus mechanism that guarantees the integrity and confidentiality of the hyperparameters of local learning models. Despite all the critical issues addressed by deploying FDL in these above-mentioned architectures, decentralized and federated learning still faces challenges such as improving the training time of learning models, security of data and learning models, data heterogeneity issue, and embedded resources on edge devices. This leads us to design a new secure and distributed federated learning architecture based on new lightweight and secure network and telecommunication technologies.