Given that many globally leading diseases like diabetes and hypertension have early manifestation in the retina, development of AI enabled low-cost non-invasive pre-diagnostic tools is of great potential benefit for mass screening especially in low and middle income countries. Despite a substantial amount of existing work in this area, very few methods have been translated into the clinic because of two reasons. First, the list of diseases detectable from retinal fundus imaging is ever increasing with new medical research, and hence the AI pipelines need to be designed in a staged semi-supervised fashion to ensure scalability and generalisation. Second, it is crucial for such models to transparently align with established clinical features in order to be adopted as clinical decision support systems (CDSS). We present RetNet, a two-stage deep neural network that addresses both issues simultaneously, by providing a computationally scalable as well as clinically traceable system. In the first stage, we use autencoders to learn the healthy class in a semi-supervised manner and then do a binary classification to identify abnormal images in an anomaly detection setting. In the second stage, we use a convolutional neural network (CNN) based classifier for the disease categorization. Instead of using the whole retinal image as input in a domain agnostic way, we use clinically relevant RoIs as inputs in parallel learning channels to ensure interpretability and trustworthiness. Results show that RetNet produces competitive results on benchmark datasets like JSIEC and RFMiD across a range of diseases.