Diabetic retinopathy (DR) is one of the significant factors of vision impairment and blindness among diabetic patients. There is a lack of early and accurate detection approaches to prevent the progression of DR severity. Traditional DR screening approaches rely on manual interventions, are prone to human error, and are challenging to scale in clinical practice. The advancement of deep learning (DL) techniques in medical imaging has led to the development of automated DR classification. However, the existing approaches encounter challenges in extracting key patterns associated with DR severities, resulting in suboptimal performance. The purpose of this study is to develop an interpretable DR severity classification, improving the trustworthiness of the model’s decision. Thus, the authors introduce a hybrid DL framework integrating RegNetX-SWIN-based feature extraction, tensor-based feature fusion, and Bayesian optimization and Hyperband (BOHB)-fine-tuned extremely randomized trees (ERT), to detect and classify DR severity levels using the fundus images. They employ a comprehensive five-fold cross-validation using the EyePACS dataset to evaluate the model’s performance. The experimental findings highlight the effectiveness of the proposed model in identifying DR severities, outperforming the existing models in terms of accuracy, sensitivity, specificity, and F1-score. The integration of Shapley Addictive exPlanations (SHAP) values facilitated the model’s interpretability, explaining the model’s decision to ophthalmologists. The findings exhibit the model’s capability in differentiating multiple severity levels, enhancing real-time deployment, and the model’s adaptability to diverse clinical environments. By enabling interpretability and transparency, the proposed model addresses the limitations in DR screening approaches. In the future, the model can be extended by incorporating multi-modality-based feature fusion capabilities.