As deep learning scales to larger datasets and more complex models, basic gradient descent can suffer from slow convergence, sensitivity to learning-rate choice, and entrapment in suboptimal regions. This chapter explores advanced optimization strategies that enhance training efficiency and stability. We begin by clarifying the roles of batch size, epoch, and iteration, and by demystifying mini-batch and stochastic gradient descent for memory-efficient learning. Next, we dive into momentum-based updates—which accumulate a velocity term to overcome oscillations and escape local minima—and derive their parameter-update equations. Building on momentum, we explore RMSprop and Adam, two adaptive–learning‐rate algorithms that automatically tune step sizes per parameter to accelerate convergence. We also discuss learning-rate decay schedules (inverse-time and exponential) for balancing initial speed with fine-tuned convergence. Practical code examples demonstrate each optimizer on toy objectives and on a real-world diamond-price regression task, showing how Adam dramatically reduces training time compared to vanilla gradient descent. Finally, the chapter introduces batch normalization—a technique that standardizes layer inputs to reduce internal covariate shift, speed up learning, and provide a mild regularization effect. By mastering these advanced methods, readers will be able to train deeper networks more reliably and achieve faster, more robust convergence.