India’s reliance on fossil fuels to meet its energy generation demands has a negative impact on both the environment and public health. In the present energy transition, photovoltaic (PV) technologies have become more widely accepted for commercial use, achieved significant progress in technology, and have a major role to play in reducing the harmful environmental effects of fossil fuel-based power generation. Soiling research on solar PV has notably increased in recent years due to its critical role in reducing the performance of solar PV cells. Therefore, to guarantee the amount of power yield, efficient monitoring of the soiling is crucial. It is difficult to manually analyse soiling severity on the panel images due to the non-uniformity of the type and amount of soiling faults. Deep learning models have been successfully applied to the analysis of solar panel images and soiling categorization; nevertheless, this approach may require a significant amount of training time and may lead to gradient descent. To tackle these training issues, we propose a comparative study of two pre-trained neural network architectures, namely, MobileNetV3small and RestNet50, on a publicly available dataset of 875 images of six solar PV fault classes, namely, clean, dust, bird-drop, electrical damage, physical damage, and snow-covered. The experimental results show that MobileNetV3small achieved a better performance of 98%. The results show that MobileNetV3small obtained a promising result of 11% higher classification accuracy than ResNet50, with an accuracy of 86.6%.