Training large language models (LLMs) presents significantly greater challenges compared to traditional language models, primarily due to the massive number of parameters involved. This chapter delves into effective strategies for pre-training LLMs, a critical phase in their development. We begin by exploring three fundamental pre-training tasks: language modeling, denoising autoencoding, and mixture-of-denoisers, with a particular focus on language modeling and its two key variants-prefix language modeling and fill-in-the-middle tasks. This provides a foundational understanding of the core mechanisms driving large language model development. To illustrate these concepts, we provide example code for training the LLaMA model. Next, we delve into various optimization settings essential for training LLMs, including batch-based training, learning rate configuration, and practical insights for their effective implementation. We then introduce scalable training techniques, including 3D parallel training, the zero-redundancy optimizer, activation recomputation, and mixed-precision training. For 3D parallel training, we conduct a thorough comparison of different parallelization strategies, including data parallelism, pipeline parallelism, and tensor parallelism, highlighting their respective strengths and trade-offs. Additionally, we address the practical aspects of running LLMs, such as calculating the number of parameters and estimating memory costs, presented in a clear, step-by-step manner. Finally, we conclude with a simplified pre-training code practice that integrates the discussed techniques, demonstrating how to construct an end-to-end pre-training pipeline for LLMs. This chapter aims to provide a basic guideline for researchers and practitioners seeking to navigate the intricate landscape of LLM pre-training.