When utilizing numerical models for decision-making in the absence of experimental data, analysts often face the challenge of extrapolating simulation predictions to untested application domains. Although this extrapolation is recognized as potentially hazardous, limited methods exist to establish the credibility of such predictions. In this chapter, we present a novel diagnostic tool for quantifying the robustness of forecasting errors in the presence of uncertainties arising from the calibration process, encompassing compensating effects and measurement errors. Our approach leverages Bayesian inference to calibrate a stochastic numerical model, which enables the incorporation of probabilistic assessments. To establish a robustness indicator, we rely on numerically generated data from both the validation and forecasting domains. Through proxy simulations that emulate the application domain, we account for the absence of experimental data and consider the uncertainties inherent to the measurement process. By combining these elements, we formulate a robustness criterion that quantifies the permissible level of calibration errors while maintaining a critical threshold for forecasting accuracy. The resulting indicator guides decision-makers on whether extrapolation should be avoided, particularly when only small errors can be tolerated. To illustrate the effectiveness of our methodology, we apply it to an aluminum frame structure, showcasing its applicability in a practical context. By utilizing validated physics-based simulations and incorporating data uncertainties, our approach provides a comprehensive framework for assessing the robustness of numerical models in forecasting scenarios. This research contributes to bridging the gap between simulation-based predictions and real-world decision-making, aiming to mitigate the inherent challenges associated with extrapolation and offering valuable insights for informed decision-making.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Elephant in the Room: Forecasting Using Validated Physics-Based Simulations

  • Rafael de O. Teloli,
  • François Hemez,
  • Scott Cogan

摘要

When utilizing numerical models for decision-making in the absence of experimental data, analysts often face the challenge of extrapolating simulation predictions to untested application domains. Although this extrapolation is recognized as potentially hazardous, limited methods exist to establish the credibility of such predictions. In this chapter, we present a novel diagnostic tool for quantifying the robustness of forecasting errors in the presence of uncertainties arising from the calibration process, encompassing compensating effects and measurement errors. Our approach leverages Bayesian inference to calibrate a stochastic numerical model, which enables the incorporation of probabilistic assessments. To establish a robustness indicator, we rely on numerically generated data from both the validation and forecasting domains. Through proxy simulations that emulate the application domain, we account for the absence of experimental data and consider the uncertainties inherent to the measurement process. By combining these elements, we formulate a robustness criterion that quantifies the permissible level of calibration errors while maintaining a critical threshold for forecasting accuracy. The resulting indicator guides decision-makers on whether extrapolation should be avoided, particularly when only small errors can be tolerated. To illustrate the effectiveness of our methodology, we apply it to an aluminum frame structure, showcasing its applicability in a practical context. By utilizing validated physics-based simulations and incorporating data uncertainties, our approach provides a comprehensive framework for assessing the robustness of numerical models in forecasting scenarios. This research contributes to bridging the gap between simulation-based predictions and real-world decision-making, aiming to mitigate the inherent challenges associated with extrapolation and offering valuable insights for informed decision-making.