Constructing from the preceding chapter, the purpose of this chapter is to provide the critical review of relevant literature to reveal the gaps in literature so that the current research turns into the complement. Initially, the study inspects the concept of optimization and determines the optimization algorithms development including both the stochastic and deterministic algorithms. Moreover, various types of optimization algorithms and their applications, strengths, and weaknesses are analyzed in this chapter. Additionally, the chapter highlights the detail background concepts of the research work, the Grey Wolf Optimizer (GWO), the neural network and its training, and the application of meta-heuristic algorithms for training neural network. The chapter also analyses literature that is relevant to ensemble approach, strength of GWO, limitation, and enhancement of GWO algorithm. Finally, the chapter provides a critical gap analysis in order to justify the proposed research work.