Scholarlink

<p>Convolutional Neural Networks (CNNs) and transformer architectures offer strengths for modeling temporal data: CNNs excel at capturing local patterns and translational invariances, while transformers effectively model long-range dependencies via self-attention. This paper proposes a hybrid architecture integrating convolutional feature extraction with a multi-head attention backbone to enhance multivariate time series forecasting. The CNN module first applies a hierarchy of one-dimensional convolutional layers to distill salient local patterns from raw input sequences, reducing noise and dimensionality. The resulting feature maps are then fed into the prediction model, which applies multi-head attention to capture both short- and long-term dependencies and to weigh relevant covariates adaptively. We evaluate the proposed CNN-transformer-based model on a hydroelectric natural flow time series dataset. Experimental results demonstrate that the proposed model outperforms well-established deep learning models, with a mean absolute percentage error of up to 2.2%. The explainability of the model is obtained by a proposed SHapley Additive exPlanations (SHAP) and Multi-Head Attention Weights (MHAW). Our novel architecture, named CNN-Transformer-SHAP-MHAW, is promising for applications requiring high-fidelity and multivariate time series forecasts.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CNN-transformer-based model explained by SHAP and multi-head attention weights for time series forecasting

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月25日

查看全文
  • Stefano Frizzo Stefenon,
  • João Pedro Matos-Carvalho,
  • Valderi Reis Quietinho Leithardt,
  • Kin-Choong Yow

摘要

Convolutional Neural Networks (CNNs) and transformer architectures offer strengths for modeling temporal data: CNNs excel at capturing local patterns and translational invariances, while transformers effectively model long-range dependencies via self-attention. This paper proposes a hybrid architecture integrating convolutional feature extraction with a multi-head attention backbone to enhance multivariate time series forecasting. The CNN module first applies a hierarchy of one-dimensional convolutional layers to distill salient local patterns from raw input sequences, reducing noise and dimensionality. The resulting feature maps are then fed into the prediction model, which applies multi-head attention to capture both short- and long-term dependencies and to weigh relevant covariates adaptively. We evaluate the proposed CNN-transformer-based model on a hydroelectric natural flow time series dataset. Experimental results demonstrate that the proposed model outperforms well-established deep learning models, with a mean absolute percentage error of up to 2.2%. The explainability of the model is obtained by a proposed SHapley Additive exPlanations (SHAP) and Multi-Head Attention Weights (MHAW). Our novel architecture, named CNN-Transformer-SHAP-MHAW, is promising for applications requiring high-fidelity and multivariate time series forecasts.