Deep learning-based semi-supervised object tracking system plays a pivotal role in the field of visual object tracking (VOT), due to its high accuracy. In the existing object tracking algorithm, object region is chosen manually. However, many computer vision applications have worked without human interference in recent days. In this consequence, we introduce a semi-supervised tracking algorithm with the help of a deep network, Customized Encoder-Decoder SegNet (CEDSegNet), and salience features, for tracking an object in videos. Specifically, deep learning model employs detection and extraction of object region, says region of interest (ROI) of an object, in a video frame. We use this ROI for estimation of the salience of an object in subsequent frames using log-likelihood. Finally, we apply the mean shift algorithm on detected object for tracking of the object. The qualitative results are obtained from the CDNet2024 dataset. The experimental results showcase the effectiveness of the proposed tracking system.