In this chapter, we study the linear regression model as a fundamental example and a cornerstone of machine learning. Two equivalent approaches—maximum likelihood estimation and least squares approximation—are used to formulate the associated optimization problem. We derive the optimal solution and discuss its key properties. In addition, we introduce the variance inflation factor as a quantitative measure of linear correlation among features.