In this chapter, we show how PCA can be developed in a space endowed with a Euclidean structure. The space studied is the vector space of matrices of a given dimension, on which PCA is run as a best approximation with a matrix of the same dimensions but lower rank. The term “best approximation” is to be understood as the approximation by the smallest distance once a Euclidean structure has been chosen via an inner product. We focus on Euclidean structures built in a consistent way with Euclidean structures in the vector space spanned by the columns and by the rows, which specify how to compare variables and items. We then develop the geometric approach to this problem, where the best projections are derived with metrics chosen on these spaces. This chapter is crucial because several of the methods presented in later chapters can be seen as PCA with specific Euclidean metrics. Thus, their solution can be built simply by translating the solution provided in this chapter.