This chapter provides a comprehensive introduction to Support Vector Machines (SVMs), a class of models that rose to prominence in the early 2000s due to their strong theoretical foundations and practical effectiveness. SVMs are grounded in the principle of structural risk minimization, which seeks to balance empirical risk–measured as classification error on training data–with model complexity. This equilibrium is achieved through regularization, where maximizing the margin between classes acts as a key mechanism to control capacity and prevent overfitting. A central theme is the concept of the margin, defined as the distance between the separating hyperplane and the closest data points from each class. The margin not only determines the robustness and generalization performance of the model but also forms the basis for the original hard margin SVM formulation, which seeks perfect separation for linearly separable data. To address real-world scenarios where data is not perfectly separable, the chapter introduces slack variables, allowing for controlled misclassifications and enhancing the model’s resilience to noise. The kernel trick is explored as a powerful technique that enables SVMs to perform non-linear classification by implicitly mapping data into higher-dimensional spaces, where a linear separator can be found. The discussion includes an analysis of the Rademacher complexity for linear classifiers in feature spaces, highlighting how the margin and the radius of the data distribution jointly influence model complexity and generalization ability. While SVMs were originally developed for binary classification, the chapter also examines their extension to multi-class problems through the Crammer-Singer method.