<p>In light of growing infrastructure and proliferation of smart devices, accurately estimating the electricity load has become crucial for several key activities such as resources management, usage planning, enhanced performance, and ensuring reliable supply. Additionally, it helps in supporting sustainability by enhancing energy efficiency and minimizing environmental impact. Traditionally, several data-driven techniques employing machine learning and deep learning methods for accurate forecasting have been proposed in the literature. However, the inherent nonlinear, complex, temporal, and stochastic nature of the consumption patterns makes the forecasting problem a challenging task. To address these challenges, the current study proposes a hybrid approach integrating a decomposition technique with a sparse self-attention-based transformer architecture that reduces the computational burden compared to standard transformer architectures for energy load estimation. The decomposition strategy allows defining the nonlinear, abrupt, and complex fluctuations of the load profile patterns, which are subsequently fed into the proposed transformer architecture. The proposed transformer architecture with sparse attention mechanism then effectively captures the intrinsic spatial and temporal dependencies patterns within the decomposed data for performing accurate predictions. The performance assessment of the proposed approach is performed on the four real-time electricity usage datasets from Southern Indian states. The evaluation results demonstrate the superiority of the prediction results (0.92–2% prediction error) achieved by proposed approach in comparison with existing deep learning-based models and state-of-the-art transformer architectures, including Transformer, Autoformer, and Crossformer. Lastly, the robustness and generalizability of the proposed approach are validated on two out-of-the domain datasets.</p>