This chapter introduces one of the simplest yet very useful types of machine learning algorithms: linear models. First, a theory for the basic linear model algorithm will be laid down. Then, the employment of regularization to generate other common types of linear model algorithms, e.g., Ridge and Lasso, will be explained. Linear models are, by nature, proposed for regression. The extension of such algorithms to classification, including both binary classification and multiclass classification, will be introduced next. Finally, we will show the use of kernel functions to turn linear models into nonlinear methods for regression and classification.