Small but mighty: enhancing time series forecasting with lightweight LLMs
摘要
While large language models (LLMs) have demonstrated remarkable potential in time series forecasting, their practical deployment remains constrained by excessive computational demands and memory footprints. Existing LLM-based methods typically suffer from three critical limitations: (1) inefficient parameter utilization in handling numerical time series patterns; (2) modality misalignment between continuous temporal signals and discrete text embeddings; and (3) inflexibility for real-time expert knowledge integration. We present small but mighty enhancing time series (SMETimes), the first systematic investigation of small language models with sub-3B parameters (SLM) for efficient and accurate time series forecasting. Our method centers on three key innovations: (1) a statistically enhanced prompt structure that bridges numerical time series with textual semantics through descriptive statistical features; (2) an adaptive fusion embedding structure that aligns temporal patterns with language model token spaces through learnable parameters; and (3) a dynamic mixture-of-experts structure enabled by SLMs’ computational efficiency, adaptively combining base predictions with domain-specific models. Extensive evaluations across seven benchmark datasets (ETTh1/2, ETTm1/2, Weather, Solar, ECL) demonstrate that our 3B-parameter SLM achieves state-of-the-art performance on five primary datasets while maintaining 3.8