Dimensionality reduction has become a commonly used part of the analysis of complex economic data. The aim of this work is to study the potential of regularized tools for dimensionality reduction methods and possibly to propose some novel regularized tools. The regularization in the form of shrinkage allows to improve numerical stability of the tools for high-dimensional data and also to reduce variability of parameter estimates at the cost of introducing bias. Firstly, a robust regularized version of the coefficient of multiple correlation is proposed, which may be exploited within a Minimum Relevance Maximum Redundancy supervised variable selection. Secondly, the ridge regularization is discussed not to bring any modification of principal component analysis; this is true also for robust versions of principal component analysis.