Understanding the rhetorical roles of sentences within legal documents is crucial for various downstream tasks, including semantic search, summarization, and case law analysis. However, the complex structure of legal case documents, coupled with the interplay of various themes, poses challenges even for human experts. In this paper, we perform data augmentation using open-sourced Large Language Models (LLMs) to automate the identification of rhetorical roles. Specifically, we explore data augmentation techniques to address class imbalance issues within datasets, introducing a novel augmentation technique called IN_Place Augmentation to mitigate linear dependency issues. Additionally, a comparative analysis of two neural network architectures, InLegalBERT and Hierarchical BiLSTM, combined with Conditional Random Fields (Hier-BiLSTM-CRF), integrating different types of sentence embeddings was conducted. We fine-tune the InLegalBERT model using our proprietary dataset and utilize the fine-tuned embeddings to train the Hier-BiLSTM-CRF model. Our evaluations demonstrate the efficacy of the fine-tuned InLegalBERT model across diverse legal contexts, showcasing significant improvements in the Land & Property domain with an increase in the weighted average F1 score from 0.729 to 0.884, and in the Criminal domain with an increase in the macro average F1 score from 0.631 to 0.817. In summary, our contributions include leveraging LLM- Mistral 7B for data augmentation, introducing IN_Place augmentation, utilizing domain-specific transformer InLegalBERT, and developing a hybrid model integrating InLegalBERT with Hier-BiLSTM-CRF.