<p>In this study, an optimal control method is developed to handle the consensus tracking control problem of nonlinear multi-agent systems in strict-feedback form. A proposed fuzzy state observer handles unmeasured states and uncertain dynamics. The control target is achieved by a dynamic programming method based on an optimal compensation term. The adaptive controller part is developed based on the backstepping technique, transferring the problem of optimal formation tracking into an equivalent optimal regulation problem. Subsequently, the optimal compensation term is designed by using the reinforcement learning method. The final control input is the adaptive controller plus the optimal compensation term. It is proved that all the signals in the closed-looped system ensure boundedness. Simulation results demonstrate the effectiveness of the proposed controller.</p>