Cardiovascular diseases (CVDs) are the ultimate reason for death across the globe and have risen 60% over the last 30 years (Linse et al. in Heliyon 9, 2023) [1]. This study investigates various machine learning (ML) algorithms and deep learning (DL) algorithms for early recognition of heart disease in individuals alongside different techniques to optimize the models and apply them to the heart disease dataset which consists of four databases: Cleveland, Hungary, Switzerland, and Long Beach VA. First, widespread machine learning algorithms like Naive Bayes (NB), Random Forest (RF), Support Vector Machine (SVM), Decision tree (DT), K-nearest neighbors (KNN), and Logistic Regression (LR) algorithms were experimented and evaluated with accuracy, recall, precision, and f1-score alongside the Stratified K-fold cross-validation. Furthermore, the same set of models each achieved better results after hyperparameter tuning by automated BayesSearchCV and feature selection with a genetic algorithm. The SVM algorithm outperformed other models by achieving 98.13% as its accuracy. On the same dataset, straightforward deep learning algorithms such as simple neural networks, multilayer perceptron, Gated Recurrent Unit networks, and Convolutional Neural Networks were tested. They achieved less accuracy than Machine learning algorithms, demonstrating that the performance of DL and ML models relies on multifarious aspects, including the complexity of the problem and the quantity of data available for training.