Analysis of time series data for classification or prediction tasks is very useful in a variety of applications including healthcare, climate studies, and finance. As big data resources have become available in many fields, it is now possible to apply extremely high-dimensional deep learning models that can model long-term temporal and spatial context. Traditional methods such as autoregressive integrated moving average (ARIMA), long short-term memory networks (LSTM), gated recurrent units (GRUs), and recurrent neural networks (RNN) have provided robust frameworks in the analysis of time series data. However, these methods have had limited success when applied to applications where long-term context is crucial. Transformer-based architectures such as GPT and BERT have emerged as a powerful method for this class of problems. In this review, we present a detailed study of the evolution of various techniques applied in time series data from classical approaches to the state of the art in deep learning systems that model long-term context. We review the transformer-based architectures that have been successfully applied to applications involving time series or high-resolution image data. We have focused on enhanced transformer architectures that can solve important challenges such as segmentation, forecasting, and classification.