In deep learning, optimizers are algorithms or tools that help fine-tune a neural network’s settings, so it makes fewer mistakes while learning. When a model returns an output for a given input, it compares the output with the actual value and updates the weight and biases to help the model’s predictions better match the real results. The objective during the training of a neural network is to find the optimal set of parameters that enables accurate predictions to be made by the model on data it hasn’t encountered before. However, the abundance of optimizing algorithms and the different parameters which can be altered makes it a tedious task to compare different optimizers. This study compares 5 different optimizers, namely, Stochastic Gradient Descent, Adagrad, Adadelta, RMSProp and Adam before and after evaluating their best learning rates and how their performance changes when they’re trained on a sample of the data versus when they’re trained on the entire dataset. The initial learning rate is the default one set in the Keras library. The best performing optimizer which was found by this study was Adadelta. The dataset used for this study is the dog breed identification dataset from a Kaggle competition.