Traffic safety violations such as not wearing seatbelts or using mobile phones while driving are main contributors to road accidents worldwide. Studies have used various techniques to detect traffic violations, still there are various limitations, including low accuracy, windshield reflections, occlusion, and color similarity. An automated system for the detection of traffic violations was proposed. The system is divided into two stages; one is a fine-tuned YOLOv11 model to detect the car windshield area and then identifies the driver region. In addition, data preprocessing techniques are applied to enhance the quality of images. In the second stage, a neural network model is utilized that employs a transfer learning mechanism with various deep learning classification algorithms, such as ResNet34, AlexNet, VGG16, and DenseNet. Two publicly available datasets are used in our study with a total number of images, from both datasets, of 1,481, taken under various lighting conditions such as sunny, cloudy, rainy, dark, and foggy. Our extensive simulation results indicated that the ResNet34 model performed the best among the other classifiers and achieved an average accuracy of 98% for the seatbelt compliance task and 99% for mobile phone usage. Benchmark comparison with existing literature indicated that the proposed lightweight model demonstrated improved results.