This paper studies the natural language processing applications in health area with the use of advanced deep learning models like RNN, LSTM, BiLSTM, or GRU for the classification of clinical text. Literature review of relevant works had looked at the role of word embedding methods distinctly in improving the performance of NLP models. In this paper, the traditional TF-IDF and GloVe methods are compared with the recent transformer-based embedding methods which include BERT, RoBERTa, and DistilBERT. The research employs a clinical transcription dataset from Kaggle, demonstrating the utility of word embeddings when training the model with keyword description retrieved from clinical texts. In addition, the work emphasizes that the use of transformer-based embedding methods can improve the performance of clinical text classification tasks, thereby improving decision-making in healthcare. The study offers evidence on the need for optimal selection of combinations of word embeddings and models for use with clinical data in text classification tasks. It is determined that BiLSTM with RoBERTa combination achieves the best performance for clinical text classification in healthcare NLP systems among other combinations interrogated.