Fish species identification has now become an essential task, especially in the study of biodiversity, fishery management, and marine conservation. The application of several deep learning models, which include a basic CNN, Residual Network, Densely Connected Convolutional Network, and Vision Transformer for the purposes of automatic fish species classification, using underwater images taken, is presented in this paper. In the training stages, data augmentation techniques were applied to increase model accuracy. The models were first trained on a truncated version of the dataset where fish species had been grouped into 17 categories to keep the number of classes low. After refinement and validation on this categorized dataset, the models were passed to the original dataset of 333 species with complexity representing real-world development issues. Using this two-phase approach helped to create an alternative model in the first development process, to prevent encountering runtime issues with a full dataset. The results indicate that data augmentation significantly enhanced the classification accuracy across all models, with the exception of the CNN, where accuracy slightly declined following augmentation in the original dataset containing 333 classes. Notably, the ViT model achieved the highest accuracy, approaching 100%, attributed to its advanced attention mechanisms that effectively manage large datasets with intricate patterns, outperforming other models.