In this paper, a novel ECG classification method based on attention mechanism of transformer model and feature extraction was proposed to detect normal and abnormal ECG signals. RR intervals, half width of R peak as well as their distributions were taken as features of ECG signals. Then, the transformer-based neural network was trained to classify imbalanced ECG signals. This research shows how to detect heart rate variability and classify electrocardiogram (ECG) data from the PhysioNet using transformer-based neural network model. The experimental results demonstrate that the proposed method achieves a good classification performance. The proposed method can be applied to distinguish between different cardiovascular diseases or autonomic nervous system disorders and to aid in early detection and diagnosis of conditions such as arrhythmias, heart failure, or diabetic autonomic neuropathy.