This chapter reviews foundational machine learning techniques that support the development of federated learning systems. It refreshes core concepts essential for understanding later chapters, while introducing the empirical risk minimization (ERM) principle as a unifying framework. The chapter begins by outlining the components of machine learning systems-data points, models, and loss functions-and explains how ERM combines them to train models. It then explores the computational aspects of ERM, focusing on gradient-based optimization methods. Statistical properties of ERM solutions are analyzed using a simple probabilistic model. Model validation is introduced as a practical tool for diagnosing overfitting, and different forms of regularization are presented, including data augmentation, loss penalization, and model pruning. Finally, the chapter explains how regularization can be used to couple learning across devices in a federated system, laying the groundwork for personalized and collaborative learning methods. Structurally, the chapter proceeds from basic machine learning principles to more advanced ideas relevant for distributed learning and concludes with exercises designed to reinforce theoretical understanding and practical skills in linear regression and regularization.