Sarcasm detection in low-resource language (LRL) becomes a unique challenge due to limited availability of annotated data and linguistic complexity. Existing works adopted machine learning and deep learning techniques for sarcasm detection in LRL. The performance of the existing models renders poor accuracy due to the lacuna in extracting contextual, local and global features from the text. To address these issues, we proposed a novel deep learning framework to detect the sarcasm text. The proposed method consists of three stages. In the first stage, the multilingual BERT (mBERT) is utilized for effective contextual word embedding for Indian languages. The embeddings generated by leveraging the mBERT, are utilized as input features for a variation of neural network architectures, which includes Feed Forward Neural Network (FFNN), Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), Convolutional Neural Network (CNN) and Multi-Channel CNN (MC-CNN). These architectures are used to determine the effectiveness in capturing the complex sarcastic expressions in texts. Finally, FFNN with softmax is used in the classification layer to detect whether the given text is sarcasm or not. FFNN architecture served as a baseline, while convolutional and recurrent architectures are evaluated for the potential performance enhancement by spatial and temporal feature extraction, respectively. The performance of the different deep learning models is evaluated on Tamil and Malayalam datasets. The experimental results demonstrate that compare to traditional methods, mBERT embeddings in combination with neural network significantly improves the sarcasm detection accuracy. Specifically, mBERT with BiLSTM achieves better accuracy of 76.61% and 83.88% in Tamil and Malayalam dataset respectively.