Irregular Multivariate Time Series (IMTS), commonly observed in domains like healthcare and finance, introduce unique challenges to the forecasting task. Existing IMTS forecasting methods primarily address interval irregularity and channel asynchronism. However, they largely overlook sparsity drift, a mismatch in sparsity patterns between the historical and forecasting windows, which disrupts temporal dependency capturing and reduces prediction accuracy. In this work, we introduce a novel indicator, Temporal Observation Density (TOD), and subsequently build History-forEcast spArsity Drift Smoother (HEADS). Specifically, we first quantify sparsity patterns in IMTS with TOD, and derive TOD bias from historical data to forecasts to reflect severity of sparsity drift. Subsequently, we build HEADS, which utilizes TOD bias for smoothing sparsity drift and acts as a plugin seamlessly attached to IMTS forecasting models. Extensive experiments present HEADS makes up to a 5.02% improvement in Mean Squared Error (MSE) on four datasets, effectively mitigates sparsity drift while serving as a universal and lightweight plugin for forecasting tasks. The code is available at https://github.com/MatthewHu0723/HEADS .