The perceptron is regarded as the pioneering work in machine learning, and it was the forerunner of neural network research. Although the perceptron had a learning function, it had various limitations, and over time it fell into decline. Later, the backpropagation method was proposed as a learning method for multi-layer neural networks. This innovative idea attracted attention as a powerful method for acquiring a nonlinear discriminant function by learning, but it also revealed a number of problems and went into decline. Deep learning, which will be introduced in the next chapter, solved these various problems and emerged as a learning method for multi-layer neural networks that can be fit for practical use. Therefore, in this chapter, as a preparation for the introduction to the next chapter, the backpropagation method and other important techniques supporting neural networks are introduced.