Up to now, we have seen PCA and PCA with metrics on rows and columns. A matrix A being given, PCA is about finding new axes and components such that the quality of representation of A by a matrix of low prescribed rank by components on its first new axis is best. PCA with metrics is about solving this problem with any Euclidean distance in spaces spanned by rows or columns of A. PCA with instrumental variables is about solving this problem when linear constraints are given on the axes and components, i.e., they live in spaces of lower dimension. The constraints on columns, for example, can be given by a second matrix, denoted B, with as many rows as A, such that the principal components of A must live in the space spanned by the columns of B. It is shown that a solution is given by first projecting the given matrix on the space spanned by the constraints on rows and columns, e.g., projecting the columns of A on the space spanned by the columns of B, and then performing the PCA of the projected matrix. This leads to three observations: \((i)\) Finding the first axis is equivalent to finding a linear combination of the columns of B with the best correlation with all columns of A, i.e., finding the best linear regression of the columns of B which explain all columns of A, \((ii)\) PCA with instrumental variables is equivalent to PLS, and \((iii)\) it is possible to combine PCA with metrics and PCA with instrumental variables, which leads to CoA with instrumental variables.