Artificial Intelligence (AI) models are widely utilized in various applications, often requiring a large number of parameters and significant computational complexity. This paper introduces an AI model, particularly a Convolutional Neural Network (CNN) model optimized with compression and quantization techniques to minimize computational resources and memory demands. The model is tailored for deployment on compact edge devices, compatible with simple hardware systems, and with low computational complexity. Arrhythmia prediction is used as a case study to assess the model’s performance. The dataset used for the Electrocardiogram (ECG) classifier is sourced from the open-access MIT-BIH arrhythmia database. The compression technique reduced the number of parameters by approximately 2.4 times and the model sizes by 9.16 times on the quantization model compared to the original model. The optimized model achieves a remarkable classification accuracy of 99.12%, coupled with the quantization process, which increased the inference speed by a factor of 23 for each ECG segment compared to floating-point execution on edge devices. These results highlight the model’s feasibility for implementation on hardware accelerators, emphasizing the design of efficient, low-precision computation units.