Early diagnosis of skin lesions is crucial due to lack of definitive treatment options, as timely and accurate classification is essential for identifying malignant lesions. This study focuses on classifying malignant and rare skin lesions using the 10,132 images from the ISIC2019 dataset, which comprises eight classes: melanoma, melanocytic nevus, basal cell carcinoma, actinic keratoses and intraepithelial carcinoma, benign keratosis-like lesions, dermatofibroma, vascular lesion, and squamous cell carcinoma. The dataset was augmented to increase its sample size. Classification was performed using both machine learning techniques (KNN, SVM, DT, RF, ANN) and deep learning models (VGG19, Inception-V3, U-Net, CNN, RNN). To address dataset imbalance, a hybrid algorithm integrating SMOTE was developed, and transfer learning was applied to the convolutional layers of VGG16 and ResNet50. The dense and flatten layers were customized according to the dataset. Feature extraction was used to enhance classification performance. Model performance was evaluated using accuracy, recall, specificity, precision, F1 score, and AUC-ROC metrics. The hybrid algorithm achieved 86.48% accuracy on the PCA test set and 88.05% on the LDA test set. This study offers an approach for classifying eight rarely studied skin lesion types. Future research could benefit from data augmentation and strategies to address class imbalance.