Classifying sensitive information is crucial for security applications like Data Leakage Prevention, but often hindered by scarce labeled data. While GAN-BERT has shown promise for low-data scenarios, its comparative efficacy against standard BERT under extended training and the impact of critical factors like unlabeled data domain alignment remain under-investigated. We empirically evaluate standard BERT fine-tuning against the semi-supervised GAN-BERT framework for this task using adapted public datasets (Monsanto, Enron) under severe low-data constraints (10 labeled examples per class). Our findings reveal that: (i) standard BERT, with sufficient training, demonstrates a powerful capacity to fit the data distribution and achieve high accuracy even with only 10 labeled examples per class, challenging the default assumption that more complex semi-supervised methods are always superior; (ii) GAN-BERT, despite faster initial convergence, plateaus earlier and incurs higher computational costs; and (iii) critically, GAN-BERT’s performance is heavily dependent on the domain alignment of unlabeled data, with a smaller, aligned corpus outperforming a larger, mismatched one. This work offers a critical qualification of GAN-BERT’s applicability, an in-depth analysis of performance trade-offs, and provides practical insights for applying NLP in resource-constrained, security-relevant scenarios.