Sperm morphology provides vital insights into functional competence, a key factor in male fertility assessment. Standard manual evaluation is often slow, time-consuming and depends on the observer, which is prone to human error. The current study aimed at developing an AI-Powered solution to evaluate sperm morphology. We used pre-trained efficientnetb0 model and fine-tuned the layers to classify normal sperm, abnormal sperm and non-sperm based on the Sperm Morphology Image Dataset (SMIDS). Our model achieved higher accuracy (84.75%) compared to DenseNet+CBAM architecture. We also included Integrated Gradients to our model that highlights the regions that has led to the prediction. We also developed the Sperm Quality Score (SQS) to turn the model’s output into a straightforward meaningful number. Our framework would pave the way for more affordable, simpler male fertility assessment