<p>Automatic inspection and monitoring of insulators in electrical power lines are crucial for ensuring a safe and uninterrupted power supply. The advancement of Deep Neural Networks has facilitated the automatic inspection and detection of insulator faults from captured images. Yet, the current research faces several challenges such as complicated backgrounds and image quality difficulties like object interference, occlusion, and blurring. Additionally, the varied appearance, type, shape and size of the insulator, along with small-sized datasets, complicates the detection and classification tasks. Convolutional Neural Network (CNN) is a shift-invariant technique used to extract feature map which is processed by a stack of convolution and pooling layers. This method is an alternative to the convolution network, which uses self-attention-based convolution-free token-distillation transformer that is well suited for small-scale datasets. In this study, two image datasets, Insulator Defect Image Dataset (IDID) and Chinese Power Line Insulator Dataset (CPLID) are merged to form a single dataset containing images with varied appearances and complicated backgrounds. Extensive experiments are performed on pre-trained CNN models, hybrid methods, vision transformers, and data-efficient vision transformers and compare their effectiveness via the metrics like precision, recall, accuracy and mean Average Precision. The results show that data-efficient vision transformer yields optimal performance with perfect scores surpasses all other CNN pretrained methods with high classification accuracy using Adam and Adagrad optimizers.</p>