Finding a social media user’s mental health has recently caught the attention of the computational linguistics field. The abundance of social media content and the excessive use of social media applications have sparked interest in this area for linguistic-based study. Because of the complexities of mental disorders, detecting mental illnesses from social media data is a difficult task. However, the emergence of numerous machine learning and Deep Learning models, as well as the availability of sample data relevant to depression for training, has enabled the development of multiple models for the early prediction of depressive symptoms in social media users. This work presents a model to determine the severity of depression in Twitter users using several context-aware embedding approaches to stimulate research in this direction. In our proposed model, we experimented with various pre-trained transformer models for domain-specific feature extraction, such as mental-BERT, bioclinicalBERT, and mental-RoBERTa. We also experimented with their stacked embeddings, and on top of these, we built various machine learning and RNN models for classification. BiLSTM combined with stacked embedding achieved the highest accuracy of 68%, followed by Mental-RoBERTa.