In this chapter, we discuss the idea of linear least squares estimation of parameters. Least squares (LS) analysis is one of the foundations of signal processing and is the fundamental concept in adaptive systems, linear prediction/signal encoding, system identification, machine learning, and many other applications. We start off with a quick look at a few applications of least squares and go on to develop the LS model. We then develop the so-called normal equations for solving the LS problem. We discuss several statistical properties of the LS solution including the Cramer–Rao lower bound (CRLB). We look at the performance of the ordinary normal equation estimates relative to the CRLB in the presence of white and coloured noise. We find that the ordinary normal equations yield optimal results in the presence of white noise. We show that in the coloured noise case performance is degraded, and so we consider various methods for whitening the noise, a process that restores the optimal performance of the LS estimator. We then move on to the important topic of adaptive filtering, which addresses the problem of efficiently computing the LS solution in nonstationary environments.