The need to automate solar cell quality monitoring techniques is growing as solar power utilization gets significant momentum. However, the detection of internal defects in photovoltaic (PV) cells is still quite a challenge, highlighting the demand for a reliable method to reduce the requirement for manual inspections. This paper presents a deep learning model to automatically detect and classify defective PV cells in Electroluminescence images using attention-based U-Net image segmentation. The proposed deep learning architecture uses ResNet50 and attention U-Net networks to formulate a set of discriminating image features, providing an effective procedure of deep transfer learning. The extracted image descriptors are encoded and learned from image masks and annotations that were reprocessed to improve the generalization capability of the photovoltaic cell classifier. The semantic deep learning model with image augmentation and mask processing provides a baseline to detect and classify any possible faults in photovoltaic cells. The experimental results demonstrate the effectiveness of semantic attention U-Net-Resnet50 deep architecture in classifying PV cells, achieving a precision of 95% and an F1-score of 94%. It also achieved an IoU of 90% for PV image segmentation.