Agriculture is the most important industry around the globe and requires technological updates to sustain both human and animal life. Crop diseases are one of the significant challenges in this industry. Recently, artificial intelligence-based deep learning architectures have become highly effective tools for the early detection of diseases in crops. Transformer models, known for their effectiveness in various tasks, have garnered attention in this field for the last few years. In this study, the vision transformer-based architecture is trained to classify the diseases in two nightshade crops, potato and tomato. The model achieved an impressive accuracy of 99.73% for potatoes and 99.59% for tomatoes with fewer parameters than the deep learning models. The publicly available dataset is used for training purposes. The proposed model’s results are compared with state-of-the-art deep learning models for validation, demonstrating that vision transformer models outperform traditional models. Overall, the findings of this study have potential applications in the agricultural area to improve potato yield and mitigate financial losses arising from misclassification.