Credit card fraud is a pervasive issue that causes significant financial loss, thus underscoring the urgent need for effective detection techniques. In this book chapter on One-Class Classification (OCC) critical issues are thoroughly examined. The first deals with the training of OCC classifiers on majority versus minority class data. Our results show that training on the majority class yields more favorable scores, with the One-Class GMM algorithm emerging as the top performer. The second issue addresses the selection of an appropriate performance metric. We demonstrate that Area Under the Precision-Recall Curve (AUPRC) is a more reliable measure than Area Under the Receiver Operating Characteristic Curve (AUC) for highly imbalanced datasets, as evidenced by the Credit Card Fraud Detection Dataset. Finally, we show that binary classification is a more effective approach for detecting credit card fraud than OCC, with CatBoost producing the best results during experimentation. Our research serves as a robust foundation for directing future researchers towards the most promising avenues for credit card fraud detection.