The dynamic and unpredictable nature of road traffic is shaped by numerous factors, including environmental conditions, human behavior, and infrastructure elements such as traffic signals, road design, and network complexity. While existing studies often overlook the critical influence of human factors, our research integrates these considerations to provide a more holistic approach to traffic management. We employed a centroid-based object tracking methodology to monitor vehicles within defined parameters, the YOLOv3 model for precise object detection from video frames, and an innovative technique for identifying vehicles traveling in the wrong direction. Experimental results demonstrate a strong correlation between predicted and actual vehicle counts, validating the system's reliability. The proposed approach combines convolutional neural networks (CNNs) with real-time object tracking to effectively manage traffic at intersections, achieving an impressive average accuracy of 93.04% across various scenarios. This method successfully tracks diverse traffic entities, including two-wheelers and four-wheelers, even under complex and congested conditions. The system is not only efficient and accurate but also cost-effective, requiring minimal hardware and infrastructure modifications. Our findings highlight the potential of advanced AI-driven methodologies to revolutionize traffic management, paving the way for smarter and more adaptive urban mobility solutions.