Training a deep neural network requires orchestrating data, architecture, loss functions, and optimization into a cohesive learning pipeline. This chapter first breaks down the components of a fully connected network—input, hidden, and output layers—and details how each neuron performs linear transformations followed by nonlinear activations (ReLU, sigmoid, tanh, Leaky ReLU), with attention to their derivatives and the vanishing/exploding gradient issues. It then clarifies the dimensionality of weight matrices, bias vectors, and activation/pre-activation tensors across layers, ensuring consistency in forward and backward passes. Building on the gradient descent foundation, the chapter presents both non-vectorized and vectorized implementations of forward propagation, backpropagation, and parameter updates for a three-layer network, highlighting the dramatic efficiency gains from NumPy-based matrix operations. It also covers best practices for feature normalization, weight initialization (He and Xavier schemes), and binary cross-entropy loss for classification tasks. Finally, we apply these methods to a real-world credit-risk dataset: normalizing inputs, training the network, visualizing the learning curve, and evaluating performance via precision, recall, and F1-score. By the chapter’s end, readers will have coded a deep network from scratch and gained the practical know-how to train and validate models on unseen data.