The current deep learning models have to deal with overconfidence issues due to overfitting and over-parameterization. Therefore, uncertainty estimation is an essential step in understanding and explaining the model’s predictions to provide warnings and help humans trust the predictions of the AI model, especially in precise high-risk tasks, low regime data, and ambiguous datasets. This paper introduces an ambiguous dataset namely the YSH dataset and proposes a novel architecture, named the Diversity Ensemble of ResNet (DEResNet), to address the challenging uncertainty and diversity in the YSH dataset. Specifically, we use the Stein Variational Gradient Descent (SVGD) algorithm for uncertainty estimation. Then we utilize a diversity mutual information (MI) loss on target data to attack the diversity problem. DEResNet can address comprehensive uncertainty both in parameter space and output space. Extensive experiments demonstrate that our model outperforms the baseline method in the YSH dataset. Furthermore, our proposal also has the potential to learn the set of solutions for an ambiguous task and help find different solutions for optimization problems. Our code is available here https://github.com/trinh-hoang-hiep/DEResNet .