In the previous work, the authors proposed an uncertainty-aware metric called the information measure of calibration flexibility (ICaF). ICaF addresses the trade-off between goodness of fit and model generalizability, and its efficacy has been proved in calibration parameter selection and model selection. This study further investigates the impact of four influential factors on the performance of ICaF through a regression example. These factors include the model form, selection of calibration parameters, prior knowledge of the system being studied, observation characteristics (i.e., the quantity and distributions of observations), and experimental uncertainty. Models with different model simplicity affect the goodness of fit of a calibrated model and the subsequent model predictions. Prior knowledge reflects initial beliefs about the underlying system. The characteristics of observations comprehensively consider the influence of quantity and distributions of observations on the calibration process. Lastly, experimental uncertainty associated with various sources is included. This study provides a comprehensive analysis by systematically varying influential factors and assesses how these factors individually influence the effectiveness of ICaF in model calibration and forecasting. These findings contribute to a collective understanding and insights into the behavior and the robustness of ICaF, which informs the design and implementation of ICaF in scientific and engineering domains.