<p>The proliferation of AI-driven applications such as autonomous vehicles, intelligent transportation systems, surveillance-based anomaly detection, and remote health monitoring solutions etc. is catalyzed by the recent advances in technologies such as the Internet of Things (IoT), wearable devices, computer vision, and high-speed networking etc. These applications leverage cloud computing to process huge amounts of data produced by them. These AI workloads have typical characteristics such as large volumes, computational intensity, non-linearity, high variance and extreme dynamicity. Forecasting the workloads would facilitate the implementation of proactive resource management strategies. Though several deep learning techniques have been used for time series forecasting in the past, LSTM variants such as Bi-LSTM, GRU, ConvLSTM and Transformers remain desirable choices for time series forecasting. The LSTM variants, as well as the transformers are originally designed for sequential processing as in Natural language processing (NLP). These models require adaptations to efficiently perform workload forecasting. This research work employs multi-resolution analysis achieved through fast wavelet transforms (FWT) to reduce the computational complexity of LSTMs. Further, the research work also investigates the performance of the FWT-enabled LSTM variants with that of Multi head attention(MA) transformer. For the experimental study, GPU cluster trace from Alibaba’s Platform for Intelligence (PAI) is used. Experimental findings show that the FWT improves the performance of GRU by around 50% and it also significantly reduces the training time. In the absolute metrics, FWT-enabled GRU outperforms the Transformer by approximately 30% in terms of Mean Absolute Error (MAE), with values of 12.8 and 18.5, respectively. However, in terms of variance explanation (R<sup>2</sup>), the vanilla Multihead attention Transformer achieved 0.1, which is higher than the GRU (FWT-enabled) model’s R<sup>2</sup> of 0.01.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-Resolution Analysis Enabled LSTMs and Transformers: AI/ML Workload Forecasting in Cloud Environments

  • S. Thiruchadai Pandeeswari,
  • S. Padmavathi

摘要

The proliferation of AI-driven applications such as autonomous vehicles, intelligent transportation systems, surveillance-based anomaly detection, and remote health monitoring solutions etc. is catalyzed by the recent advances in technologies such as the Internet of Things (IoT), wearable devices, computer vision, and high-speed networking etc. These applications leverage cloud computing to process huge amounts of data produced by them. These AI workloads have typical characteristics such as large volumes, computational intensity, non-linearity, high variance and extreme dynamicity. Forecasting the workloads would facilitate the implementation of proactive resource management strategies. Though several deep learning techniques have been used for time series forecasting in the past, LSTM variants such as Bi-LSTM, GRU, ConvLSTM and Transformers remain desirable choices for time series forecasting. The LSTM variants, as well as the transformers are originally designed for sequential processing as in Natural language processing (NLP). These models require adaptations to efficiently perform workload forecasting. This research work employs multi-resolution analysis achieved through fast wavelet transforms (FWT) to reduce the computational complexity of LSTMs. Further, the research work also investigates the performance of the FWT-enabled LSTM variants with that of Multi head attention(MA) transformer. For the experimental study, GPU cluster trace from Alibaba’s Platform for Intelligence (PAI) is used. Experimental findings show that the FWT improves the performance of GRU by around 50% and it also significantly reduces the training time. In the absolute metrics, FWT-enabled GRU outperforms the Transformer by approximately 30% in terms of Mean Absolute Error (MAE), with values of 12.8 and 18.5, respectively. However, in terms of variance explanation (R2), the vanilla Multihead attention Transformer achieved 0.1, which is higher than the GRU (FWT-enabled) model’s R2 of 0.01.