<p>Automated classification of medical abstracts is critical for managing the vast and rapidly growing body of biomedical literature, but it requires models that can comprehend complex, domain-specific language. While transformer-based models like BERT have proven effective, the relative performance of general-purpose versus domain-specific models remains an important consideration. This study presents a comprehensive comparative evaluation to address this issue. Six prominent transformer models, including BERT-base, RoBERTa, DistilBERT, SciBERT, BioBERT, and ClinicalBERT, were fine-tuned and evaluated on two standard benchmarks: the Ohsumed and PubMed 20k RCT datasets. Performance was primarily assessed using the weighted F1-score to account for class imbalance. The results consistently demonstrate that domain-specific models outperform their general-purpose counterparts. On the Ohsumed dataset, the top-performing model, SciBERT, achieved an F1-score of 81.69%, a significant improvement of over 6.4% points compared to the BERT-base baseline. Notably, the optimal model was found to be dataset-dependent, with BioBERT achieving the highest F1-score of 86.77% on the more structured PubMed 20k RCT dataset. The findings conclude that while domain-specific pre-training provides a distinct advantage, the optimal model choice is contingent on the linguistic characteristics of the target corpus, highlighting that a “one-size-fits-all” approach is suboptimal for medical text classification.</p>