In the previous chapters we considered the LS problem where \(\boldsymbol {X}\) is full rank and tall. In many practical problems \({\boldsymbol {X}}\) is often poorly conditioned or rank deficient. We have seen previously that in these cases the performance of the ordinary normal equation solution to the LS problem can diminish drastically. In this chapter we consider various methods for mitigating this problem. The first is the pseudo-inverse method, which is based on the SVD and has proven effective in solving the rank-deficient LS problem. We also show that the pseudo-inverse approach is intimately connected with principal component analysis in this context. These methods control the large variances of the LS solution that occur in the near rank-deficient case. We then apply the rank-deficient QR decomposition for solving the rank- or near-rank-deficient LS problem. We also develop additional LS techniques, referred to as latent variable methods, which are specifically designed for the near-rank-deficient case. These are the principal component analysis (PCA), partial least squares (PLS), and canonical correlation analysis (CCA) methods, respectively. Further, we show the equivalence of the PCA and the pseudo-inverse methods.