Object tracking has become a cornerstone of modern computer vision systems, with applications ranging from intelligent transportation to public safety and retail analytics. Classical trackers—such as the Channel and Spatial Reliability Tracker (CSRT) and ORB feature-based methods - offer robust object localization and tracking but are inherently limited to predefined classes. Recent advances in Zero-Shot learning enable detectors to identify arbitrary classes without retraining by leveraging visual–semantic alignment. This paper proposes a hybrid tracking framework that combines a Zero-Shot detector with ORB feature matching and CSRT tracking to tackle the challenge of tracking arbitrary object classes. The system is validated on a custom video dataset recorded near a bakery in Amsterdam for pedestrian counting. Results show that integrating all three components yields superior F1-Score and MOTA compared to baseline methods. Beyond accuracy, we emphasize deployment practicality: the pipeline uses a single modern GPU for inference, runs at near-real-time rates on 1080p footage, and maintains stable performance across varying crowd densities and lighting conditions. The proposed approach can be deployed in smart city infrastructure, retail customer-flow analytics, traffic monitoring, and surveillance in public spaces. Its ability to handle unseen object categories without retraining makes it highly adaptable to dynamic environments. The modular design further allows components to be swapped (e.g., alternative captioners or open-vocabulary detectors) without re-engineering the full stack.