The analysis of retinal fundus images enabled by AI for the diagnosis of early disease is one of the popular grand challenges in clinical machine learning, but it still remains an open problem as medical literature continues to discover new diseases that have early visual manifestations in the retinal image. The solution therefore is to leverage the recently developed large vision transformer based foundation models to generate rich generalized feature embeddings from retinal images in a self-supervised manner. These embeddings can then be utilized for downstream disease detection through smaller easily retrainable models should new classes (disease types) be continually added down the line. The catch here is that the foundation models due to their size and millions of pre-trained weights are essentially black boxes, and hence difficult to be trusted in clinical practice for patient safety. In this work, we aim to bring these two seemingly conflicting ideologies to a satisfactory synergy by making the foundation model (existing RETFound) focus attention on clinically relevant areas of interest in the fundus images for transparency (proposed ROIRETFound), so as to exploit the power of foundation models and at the same time maximize the chances of clinical translation through explainability. The proposed approach achieved an average F1 score of 0.91 and 0.86 in disease detection on two publicly available datasets JSIEC and RFMiD, outperforming standard CNN benchmarks (ResNet, VGGNet) as well as a recent competitor RetNet.