Due to the widespread expansion of digital content on social media platforms, there has been a substantial increase in the research interest on Sentiment Analysis (SA) of social media text in many south Asian languages including Hindi, Bangla, Tamil, Kannada, etc.. Despite the development of numerous SA models, the evolving nature of user-generated content which is becoming more diverse and creative, underscores the need for more efficient learning models. Added to this, the increasing number of social media users and the increasing amount of text generated by these users, demand the automated tools for SA. To address the challenges of SA in Bangla - an under-resourced language, in this paper, we explore a wide range of transformer models that support Bangla: Bangla Bidirectional Encoder Representations from Transformers (BanglaBERT), BanglaBERT-base, Bengali-BERT-Base, Multilingual BERT (mBERT), Multilingual Distil BERT (mDistilBERT), Cross-lingual Language Model Robustly Optimized BERT Pretraining Approach (XLMRoberta), Multilingual Decoding-enhanced BERT with disentangled Attention (mDeberta), and indic-BERT, using Transfer Learning (TL) to classify the given Bangla text into one of the three classes - ‘Positive’, ‘Negative’, or ‘Neutral’. Further, to overcome the data imbalance issue in the dataset, text augmentation techniques are explored using two libraries: NLPAug and bnaug, to increase the number of samples in the minority class. Among all the transformer models, BanglaBERT model fine-tuned with the Train set augmented by using techniques in NLPAug library achieved a micro F1 score of 0.8203.