<p>We present a new multiclass classification method based on hidden Markov model (HMM). This method consists of identifying the posterior probabilities that belong to each class by first exploring the estimation techniques of HMM to build a robust probabilistic classifier. Then, an iterative algorithm for computing the left inverse is developed and implemented in the learning process of the classifier to enhance the applicability of the proposed approach. We experiment with various schemes of initialization of left inverse calculation and investigate their impact on the predictive performance of the classifier. Numerical experiments on both artificial and real datasets demonstrate that the HMM combined with the bootstrap technique outputs the estimate of probability well, in comparison to the standard approach as presented in benyacoub et al. Experiment results show that the proposed approach achieves competitive test accuracies while also produced concise models, in terms of performance metrics and established better generalization ability.</p>