Model comparison is a fundamental step in model selection, aimed at comparing the performance of two models using a commonly-used metric such as the \(\textrm{F}_1\) measure. Bayesian model comparison offers a principled framework to quantify the performance superiority of one model over another based on the averaged cross-validated estimator of \(\textrm{F}_1\) measure. However, existing comparison methods may yield inaccurate quantifications due to significant deviations between the distributions of averaged cross-validated estimators and the true distribution of \(\textrm{F}_1\) measure. To overcome this limitation, we propose a novel Bayesian model comparison approach based on an improved cross-validated estimator. Our approach constructs the voted cross-validated estimator of \(\textrm{F}_1\) measure with small distributional deviation to improve the existing averaged estimators by leveraging the block-regularized \(m {\times } 2\) cross validation coupled with a majority voting technique. Building upon the voted estimator, we compute Bayes factors to quantify the superior performance of one model over another and perform Bayesian model comparison. Experimental results demonstrate that our voted estimator exhibits an obviously smaller distributional deviation compared to the averaged estimators of existing methods. The resulting Bayes factors provide more accurate quantification of performance superiority in model comparison. These findings contribute to advancing model selection methodology and provide valuable insights for developing accurate model comparison methods.