Gradient descent is the fundamental algorithm for training neural networks by iteratively minimizing a differentiable loss (or cost) function. This chapter begins by building intuition for gradient descent through the simple problem of locating the minimum of a quadratic function, then connects those principles to parameter optimization in machine learning. Using logistic regression as a prototypical “one-neuron” network, we unpack the concepts of forward propagation, binary cross-entropy loss, and the aggregated cost over a dataset. We derive the gradient-based update rules—showing how the learning rate controls the size of each step—and distinguish between individual-sample losses and the overall cost function. To solidify understanding, we dive into a non-vectorized Python implementation of gradient descent for logistic regression: computing activations, losses, and parameter gradients via explicit loops and the chain rule (backpropagation), and performing a single update step. We then look at how to leverage NumPy’s matrix operations to perform the same computations more efficiently in a fully vectorized form, eliminating explicit loops over examples. By walking through both non-vectorized and vectorized implementations from scratch, readers gain practical insight into the mechanics, computational trade-offs, and code patterns that underlie gradient-based optimization in deep learning models.