Regularized regression techniques, like Ridge and Lasso regression, are extensively employed in machine learning for precise data analysis and forecasting. In the big data landscape, privacy concerns are more critical than ever, especially regarding the potential for data breaches. Secure multi-party computation (SMPC) presents a viable solution, facilitating collaborative machine learning projects without compromising the security of private data. This paper proposes two privacy-preserving algorithms tailored for regularized regression in a two-party context: Ridge and Lasso regression. The outlined approach for Ridge regression utilizes secret sharing protocol, multiplication triple protocol, and secure computation primitives. For Lasso regression, the proposed algorithm integrates an additional plain proximal gradient descent technique, significantly reducing computational and communication overhead. Both algorithms are proven to be UC-secure against semi-honest adversaries. Experimental results underline their effectiveness, demonstrating that they can achieve an accuracy comparable to that of plaintext models. Compared to other similar algorithms, our algorithms have significant advantages in terms of both running time and communication overhead.