Instead of randomly acquiring training data, Uncertainty-based Active Learning (UAL) selects pivotal samples from an unlabeled dataset based on the prediction uncertainty and queries their labels so that the labeling cost for model training can be minimized. As a result, the efficacy of UAL depends on the model capacity as well as the adopted uncertainty-based acquisition function. In this study, our analytical focus is directed toward comprehending how the capacity of the machine learning model may affect UAL efficacy. Through theoretical analysis and comprehensive simulation and empirical studies, we demonstrate that UAL can lead to worse performance compared to random sampling when the machine learning model class has low capacity and is unable to cover the underlying ground-truth.