<p>In developing nations, agriculture is the backbone of its economy. In addition, it has a significant impact on supplementary livelihoods and the society. Plant diseases affect the lives of many farmers. Leaf diseases have dangerous consequences on plants as it reduces the amount, productivity, or quality of produced crops. As the traditional approaches are expensive as well as time-consuming, accurate and fast diagnosis is necessary for managing and controlling plant diseases. Depending on the quality of the retrieved features, deep learning-based technologies can identify plant diseases with high accuracy. Hence, in this paper, an accurate image-based plant disease classification method with severity estimation is developed to support farmers in increasing agricultural productivity. The main aim of this work is to develop an effective identification system for plant diseases and also estimating the severity of the diseases to take preventive measures for decreasing the disease spread. The images required for the classification of diseases is collected from the benchmark datasets. The collected images is applied to the Atrous Spatial Pyramid Pooling-based Trans-Unet++ (ASPP-TransUnet + +) model for the segmentation of images. The disease-affected region of the images is effectively identified through this segmentation approach. The disease classification is done with the implementation of an Adaptive Efficient Attention Network (AEANet) with higher accuracy. The classification efficiency is further improved by the optimal tuning of parameters from AEANet, which is done with the help of the Fitness-based Secretary Bird Optimization Algorithm (F-SBOA). Finally, a severity assessment is made to choose the appropriate pesticides and take relevant measures to cure the plant disease. The plant disease classification and severity assessment efficiency is validated with the conventional models to ensure performance.</p>