In this article, we analyze the use of transformers such as DistilBERT for the binary classification of biomedical articles and text along with its potential usage in combination with semidefinite programming cut algorithms such as the Goemans-Williamson Algorithm for MaxCut. We offer an improved performance in vectorizing text for other data clustering tasks and an improvement in binary classification using transformer-based techniques to group datasets into two clusters through thorough experimentation.