Method for cardiac abnormal event recognition based on multimodal spatiotemporal feature fusion model
摘要
Electro Cardio Gram (ECG) is a crucial tool for diagnosing and treating key cardiovascular diseases such as arrhythmias and Myocardial Infarction (MI). However, most existing research focuses on One-Dimensional (1D) ECG signals, neglecting the potential benefits of integrating multiple ECG modalities to enhance analysis. This paper introduces the multimodal spatiotemporal feature fusion model, a Multimodal Fusion ConvNeXt and Transformer model (MFCT), designed to learn both temporal and spatial features of ECG signals. In the model architecture, we convert the original 1D ECG data into Two-Dimensional (2D) ECG images using the Recurrence Plot (RP) techniques, which serve as inputs to the ConvNeXt module to capture spatial features. The original 1D ECG signals are directly fed into the Transformer module to extract temporal features. The multimodal feature fusion module effectively combines these features, leading to more accurate ECG signal classification. To validate the effectiveness of MFCT, we conducted experiments on the MIT-BIH Arrhythmia Database and tested MI classification on the PTB Diagnostic Database. The results demonstrate that MFCT achieved accuracies of 99.27% and 99.17% in arrhythmia and MI classification tasks, respectively, achieving competitive performance compared to existing methods and showing advantages in overall accuracy. This highlights the potential of multimodal fusion in improving ECG signal analysis performance.