In this chapter, we go over the structure of neural networks, in enough detail to write weight gradient code. Here the neural networks need not be layered, they are allowed any topology. Then we study gradient descent methods, and use them to train neural networks. We also analyze stochastic gradient descent, and gradient descent with momentum.