Sarcasm is an emotive expression style in which a person writes or says something hilariously ironic and completely out of context. Due to the lack of corpus, very few researches have been done in this field in Bangla, even though recognized as the seventh most popular language worldwide. Addressing this gap, a dataset has been constructed named “Sarcasm-Bangla”, consisting of 5000 Bangla texts obtained through web-scrapping from various online sources such as YouTube comments, Facebook posts, newspaper headlines, etc. This study focuses on tailoring linguistic challenges by capturing complex or context-dependent nuances in sarcasm detection. For this purpose, a thorough comparative study has been conducted using our dataset to identify sarcasm in Bangla texts applying three different strategies. The first approach, a hybrid model of Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM), showed similar accuracy and f1-score of 68%. Furthermore, in the second approach, while both Logistic Regression (LR) and Multinomial Naïve Bayes (MNB) models attained highest accuracy of 73%, only LR performed well with 72% f1-score considering bigram and trigram features. Finally, the proposed model, a fine-tuned BanglaBERT with task-specific layers, achieved superior performance with 83% accuracy and 82.55% f1-score. Through our findings, it has been exhibited that the proposed approach based on BanglaBERT, outperforms both hybrid deep learning and conventional models used in machine learning. This work also addresses the robustness of the framework in terms of other existing strategies for sarcasm detection in the Bangla language.