Managing default risk in consumer loan is one of the major challenge for bankers. While, the ability of financial institutions to project in commercial loans becomes an important conerns. In this article, we present a Hidden Markov Model with multiple observable sequences revealing borrower’s information, and assuming that sequences are driven by a common hidden sequence. Efficient estimation method including the regularization effect is then used to estimate the model parameters. Therefore, the hidden risk state process based on the regularized HMM model is developed in this work to assess probability of default. Several benchmarking credit data is adopted to show the advantages of the new approach and obtained results are compared with those from the traditional HMMs. Finally, the experiment results show that the adapted form of regularization used for parameter estimation possesses a significant impact to improve the performance of HMM models.