The present work proposes an improved ensemble deep learning architecture, which aims to classify retinal fundus images into four major ocular diseases such as glaucoma, cataract, diabetic retinopathy, and normal. The architecture is formed by combining the two networks, DenseNet201 and Attention U-Net. While the dataset used had inherent class imbalances, the trained model achieved an impressive overall accuracy of 98.3. In order to enhance generalization and avoid overfitting, the model embraced the use of techniques such as brightness changes, random rotation, and zooming out of the images. In order to optimize learning, various regularization approaches were applied including but not limited to dropout, L2-norm regularization, class weight adjusting, and learning rate scheduling and stopping strategies such as ReduceLROnPlateau and EarlyStopping. The model was successful in diagnosing diabetic retinopathy and glaucoma and identified cataracts with high accuracy while also detecting healthy eyes reliably. In the face of all these challenges, the model managed to surpass several other models, as it was able to find the right balance between generalization and accuracy, concurrently preventing the issue of overfitting which is associated with better results that have been published in the literature. This research delineates the steps that can be taken for further improvement and illustrates how attention mechanisms and feature extraction can be useful in the detection of ocular ailments.