<p>Traditional evaluations of Earth system models focus on a limited set of climate diagnostics to capture large-scale energy, carbon, and mass balances, major modes of variability, and anthropogenic emissions response. As the model user base expands, there is a growing need for evaluation metrics directly aligned with climate impacts on nature and society, as well as climate services applications. We propose a broad, practical framework for impacts-relevant model diagnostic evaluation by comparing benchmark climatic impact-driver indices (CIDs) across model versions. This framework explores how shifts in model components, parameters, and resolutions may cause unintended consequences across climate conditions that drive climate impacts. Using the NASA Goddard Institute for Space Studies ModelE as a test case, we demonstrate the framework by analyzing temperature, precipitation, wind, snow, open ocean, coastal, and radiation diagnostics from eight model configurations, comparing them to observations and in their response to historical anthropogenic forcing, and evaluating how informative various CIDs are for distinguishing simulated climates amongst model versions. We find that differences in physical parameters, experimental setups, and structural components create performance tradeoffs. For instance, Model E2.2’s improved resolution and updated upper troposphere physics introduced a surface cold bias, leading to a much higher number of simulated frost days than observed. Model E3, with improved moist physics, overestimates heavy precipitation events over Africa. In some cases, CID behavior remains similar despite model differences, as seen in the lack of heavy rainfall events in South America across most versions. The framework is designed to be applicable to model evaluation across modeling centers, with the goal of systematically identifying strengths and biases associated with specific model features that are relevant to society and their changes with climate.</p>