This study highlights the importance of selecting appropriate imbalance mitigation technique and classifier for imbalanced datasets. It examines the effectiveness of various sampling techniques, cost-sensitive and hybrid ensemble approaches for improving classifier performance on imbalanced datasets using Support Vector Machine (SVM) and Logistic Regression (LR) models. The experiments conducted on the benchmark binary-class datasets with varying degrees of imbalance compare undersampling, oversampling, hybrid sampling, cost-sensitive, and hybrid ensemble methods by analyzing their impact on ROC-AUC scores. Results show that SVM generally outperforms LR, with hybrid sampling yielding significant improvements in ROC-AUC scores across three out of the five imbalanced datasets considered.