Fundamentals of Federated Learning: Principles and Applications
摘要
So far, we have motivated Federated Learning as a technique facilitating the construction of machine-learning models exploiting data held by a multitude of entities without data exchange. We have noted that what enabled Federated Learning is the consideration that we are willing to dispense with a central server preparing a single model with global knowledge about the objective of the training task. Instead, we have decided to trust the judgment of each one of the entities holding data to prepare a local model and then combine such models into an average model that is used to accomplish the training task. Federated Learning has emerged as a natural extension to distributed solutions of the seismic shift taking place in machine-learning algorithms and frameworks from both the research and industrial point of view. Although Federated Learning has brought unique challenges to the table, such as having local models run in a non-convex domain and partial execution of the training task in the entities, its formalism requirements share plenty with those of more classical distributed implementations. For instance, existing frameworks for distributed optimization methods can be easily adapted to accommodate some Federated Learning scenarios. One can expect this trend to develop even further in the future, favoring new trends, such as on-device continuous learning, in which the training task is conducted in stages over time, say, to account for the temporal non-stationarity of the estimation problem. Such new trends will also benefit from the creative implementation of Federated Learning over architectures that can be envisioned today, such as novel types of edge cloud systems, with a single layer of edge computational platforms receiving tasks from a multitude of devices.