In this chapter, we venture into a new realm of machine learning: constructing highly complex models that contain multiple layers of many nodes embedded with nonlinear activation functions and connected by linear weights, namely, feedforward neural networks. At the core of these complex models is a simple learning mechanism that we will discuss. We will also look at various treatments in these models that aim to cope with the potential high costs in training and proneness to overfitting.