The deep learning participates in many areas of human activity. It solves many routine tasks such as pattern recognition, object detection, time series prediction, and others. To increase the performance, the researchers have to develop new advanced tools, which accelerate training and raise the quality of work. One of these tools is loss function minimization. The state-of-the-art optimization algorithm with exponential moment estimation is designed to solve this problem. However, they cannot handle vanishing and exploding gradient issues and require many iterations to attain the loss function global minimum neighborhood. We proposed advanced optimizers with positive–negative moment estimation. They avoid local minimum, solve the vanishing and exploding gradient problems, and attain the global minimum for a smaller number of iterations than known analogs. Next, we integrated our optimizers into convolutional neural networks for solving the pattern recognition problem on unmanned aerial vehicles. We used the images from the University of California Merced dataset containing 21 object classes.