Scholarlink

<p>Operational aviation forecasting needs accurate predictions and probabilities for risk-based decisions. This paper examines what happens to probability calibration when a model trained in one operational weather regime is used in another, and how much target data is needed to repair it. The study uses 15 years of METAR reports from 51 airports. Six operational weather regimes are defined, and gradient boosting models are trained for four airport hazards. In the source regimes, Platt scaling worked well: NLL was between 0.040 and 0.117, and ECE stayed below 0.004. This did not carry over to new regimes. In zero-shot transfer, NLL increased in all 120 transfer-task combinations, and median ECE became roughly seven to eight times larger. To see where the recovery comes from, we used a <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(2\times 2\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>2</mn> <mo>×</mo> <mn>2</mn> </mrow> </math></EquationSource> </InlineEquation> ablation: the base classifier was either kept fixed or fine-tuned, and the Platt layer was either kept fixed or refitted. Refitting only the two Platt parameters closed about 87% of the calibration gap with 5% target data. AUC was different. After fine-tuning, median AUC recovery was about 66% at 5% target data. Thus, the two parts of the problem did not recover at the same rate. In 97 of the 120 transfer tasks, the calibration gap shrank more than the AUC gap. This was also the case when results were grouped by prevalence shift, by source and target zone, and by alternative airport groupings. Finally, we report ROC-AUC, PR-AUC, F1, and feature-importance results from both XGBoost and ridge logistic regression.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Data-efficient calibration transfer across operational weather regimes for probabilistic airport hazard prediction

  • 期刊论文

  • 发表日期：

    2026年7月6日

查看全文
  • Mustafa Semih Sadak

摘要

Operational aviation forecasting needs accurate predictions and probabilities for risk-based decisions. This paper examines what happens to probability calibration when a model trained in one operational weather regime is used in another, and how much target data is needed to repair it. The study uses 15 years of METAR reports from 51 airports. Six operational weather regimes are defined, and gradient boosting models are trained for four airport hazards. In the source regimes, Platt scaling worked well: NLL was between 0.040 and 0.117, and ECE stayed below 0.004. This did not carry over to new regimes. In zero-shot transfer, NLL increased in all 120 transfer-task combinations, and median ECE became roughly seven to eight times larger. To see where the recovery comes from, we used a \(2\times 2\) 2 × 2 ablation: the base classifier was either kept fixed or fine-tuned, and the Platt layer was either kept fixed or refitted. Refitting only the two Platt parameters closed about 87% of the calibration gap with 5% target data. AUC was different. After fine-tuning, median AUC recovery was about 66% at 5% target data. Thus, the two parts of the problem did not recover at the same rate. In 97 of the 120 transfer tasks, the calibration gap shrank more than the AUC gap. This was also the case when results were grouped by prevalence shift, by source and target zone, and by alternative airport groupings. Finally, we report ROC-AUC, PR-AUC, F1, and feature-importance results from both XGBoost and ridge logistic regression.