<p>The increasing size and complexity of pre-trained language models have demonstrated superior performance in many applications, but they usually require large training datasets to be adequately trained. Insufficient training sets could unexpectedly lead the model to overfit and fail to cope with complex tasks. Large language models (LLMs) trained on extensive corpora have prominent text generation capabilities, which improve the quality and quantity of data and play a crucial role in data augmentation. Specifically, distinctive prompt templates are given in personalised tasks to guide LLMs in generating the required content. Recently, promising retrieval-based techniques have further enhanced the expressive performance of LLMs in data augmentation by introducing external knowledge, enabling them to produce more grounded data. This survey provides an in-depth analysis of data augmentation in LLMs, classifying the techniques into Simple Augmentation, Prompt-based Augmentation, Retrieval-based Augmentation, and Hybrid Augmentation. Additionally, we conduct extensive experiments across four techniques, systematically compare and analyse their performance, and provide key insights. Following this, we connect data augmentation with three critical optimisation techniques. Finally, we introduce existing challenges and future opportunities that could further improve data augmentation. This survey provides researchers and practitioners of the text modality with avenues to address data scarcity and improve data quality, helping scholars understand the evolution of text data augmentation from traditional methods to the application of human-like generation and agent search in the era of LLMs.</p>