Deep Neural Networks (DNNs) typically require extensive datasets for effective training. In the medical domain, acquiring large-scale data is often challenging due to privacy concerns and the rarity of certain diseases. To address this data scarcity, we investigate the efficacy of training DNN models using synthetic data, generated based on domain-specific medical knowledge. Specifically, we develop a synthesis algorithm to create data for four abnormal electrocardiogram (ECG) classes: atrial fibrillation (AF), atrial flutter (AFLT), premature ventricular complex (PVC), and Wolff-Parkinson-White Syndrome (WPW). We evaluate the utility of this synthetic data by conducting abnormal ECG classification using ten different DNN architectures. Our results demonstrate that models initially trained on synthetic ECG data, followed by training on real-world data, achieve up to a 33% improvement in classification performance compared to models trained solely on real-world data. Further analysis reveals that the performance enhancement from synthetic data is more pronounced with smaller real-world datasets. These findings suggest that synthetic data is a viable alternative for training medical DNN models, particularly in scenarios where real-world data is limited or difficult to obtain, thus presenting a promising approach for advancing disease classification methodologies.