Classification of electrocardiogram (ECG) plays a critical role in the realization of smart cardiovascular surveillance systems. This paper proposes a novel hybrid framework which consolidates signal enhancement, deep learning and model compression to be able to have real-time accurate and efficient classification of ECGs. To be more specific, adaptive denoising of the signal is achieved by using a hierarchical Kalman filter, which is followed by extracting deep features with the help of an attention-based Convolutional Neural Network (CNN). Knowledge distillation is used to decrease the computational burden without sacrificing performance too much, by having a lightweight student model trained based on a heavier and more accurate teacher model. The ECG dataset is preprocessed and the model is trained on the preprocessed data and is balanced using both oversampling and undersampling methods. The offered method excels in the task of classification with a much smaller number of parameters, therefore, being very well-suitable to the areas with few resources, like wearable medical devices. This architecture will benefit signal quality, real-time deep learning inference, and deployment and has shown application in scalable, portability cardiac monitoring systems.