Model Evaluation and Benchmarking
摘要
Benchmarking time series models is challenging due to non-stationarity, irregular sampling, horizon-dependent behaviour, and structural variability across domains. This chapter develops a principled framework for evaluating temporal models, integrating classical error measures with metrics that quantify temporal alignment, distributional fidelity, and dynamical consistency. We review techniques ranging from Dynamic Time Warping, Fréchet and Wasserstein distances, and energy statistics to continuous-time criteria derived from stochastic processes and differential-equation-based dynamics. Beyond individual metrics, we examine evaluation protocols–including rolling-window, expanding-window, and multi-horizon schemes–that characterize generalization under realistic deployment scenarios. We also survey benchmark datasets designed to probe diverse temporal regimes, from high-frequency signals to sparse, event-based processes. By grounding these methods in statistical theory and dynamical systems analysis, the chapter articulates a transparent and reproducible benchmarking philosophy that strengthens comparative evaluation and advances the scientific maturity of continuous-time forecasting.