Artificial intelligence (AI) is deep-rooted and indispensable in healthcare through developing applications for diagnosis, prognosis, treatment planning, etc. Notably, in the specialization of ophthalmology, AI models have been employed to resolve critical issues in diagnosing diseases like diabetic retinopathy, age-related macular degeneration, retinopathy of prematurity, glaucoma, etc. These AI models have also been versatile in training and learning from the images and signals acquired using different modalities like fundus camera and Optical Coherence Tomography (OCT). However, there are challenges in OCT images that limit the practical application of deep learning algorithms, which are: (i) Black-box attributes: The fundamentals of deep learning models are not transparent, and, for the same reason, it can be challenging for healthcare professionals to rely on medical decisions. (ii) Poor specificity: Previous studies have been hampered by the need for more accuracy for clinical applications, especially in complex diseases. (iii) Complexity and Energy Usage: Current OCT classification deep learning models are large and computationally demanding, leading to practice challenges. The proposed research work keenly focused on mitigating the shortcomings by introducing a new practice with transfer Learning over VGG16, and independent interpretability tools like SHAP/LIME. This research article emphasizes the deep learning model, which is designed and developed exclusively for OCT images to classify and predict Choroidal Neovascularization (CNV), Diabetic Macular Edema (DME), drusen, and healthy (Normal). The proposed DL model with SHAP/LIME becomes an interpretable model that provides consistent accuracy (99.69%) with the baseline VGG16 and outperforms existing state-of-the-art methods. Using interpretability tools enables clinicians and practitioners to get the reason for the prediction, supporting their original decision.