The textile industry has grown significantly in automation in past few decades particularly in finding manufacturing defects. The purpose of this paper is to provide a Deep Learning approach to textile defect detection that makes use of Convolutional Neural Networks (CNNs) and pre-trained models such as EfficientNetB3 to effectively identify defects in textile images. In order to improve the classification performance, the implemented model approach is trained on a dataset that contains texture-based features like Gray-Level Co-occurrence Matrices (GLCM) and original images. The proposed model shows effective defect detection with a high accuracy level, reaching 92% accuracy on the test set. Moreover, the system’s performance is checked using evaluation metrics of ROC curves and AUC scores. These results along with integration of Explainable AI (XAI) using Grad-CAM adds to the trust and shows the effectiveness of proposed methodology in this paper.