The main goal of learning is to make use of the available training samples, to form an inference engine for prediction. The underlying assumption is thus hinged upon the fidelity of unseen data’s distribution with respect to the training data’s distribution. In general, those learning score functions discussed in Chap.  4 can be adopted to form the cost or criterion functions for regression predictor or classifier learning. We shall focus on several foundation analytic learning and estimation methods in this chapter.