In this chapter, we explain the intricacies of language modelling, focusing on the evolution from statistical models to the sophisticated large language models (LLMs) that dominate the field today. We explore the transition from n-gram models to neural network-based approaches, highlighting key advancements such as Word2Vec, ELMo, BERT, and the Transformer architectures. The chapter emphasizes the significance of scale in LLMs, discussing how increased model size enhances their capabilities, including context understanding and emergent behaviour. We also address the challenges associated with pre-training and fine-tuning these models, providing insights into data requirements, structural adaptations, and the implications of scaling laws. Finally, we examine the impact of model scale on predictive mechanisms and the tendency for hallucinations, proposing potential solutions to mitigate these issues.