<p>Multi-object tracking is important for applications such as surveillance and behavioral analysis. The existing approaches face challenges in identification, accurate localization, and tracking, especially in dynamic and noisy environments. To address these limitations, an efficient approach named Graph Sparse Guided Deformable Transformer-based Deep Reinforcement Q-Network is proposed for efficient identification and tracking of objects presented in different video frames. Here, different video frames were collected from various sources and are preprocessed for enhancing image quality and reducing overfitting issue. The Graph Sparse Neural Network is proposed for the extraction of essential features by utilizing both local and global networks. Then, the Guided Deformable Attention transformer is used for enhancing discriminability of the extracted features and the motions of the objects are determined based on the evaluation of trajectories. The Deep Reinforcement Learning-based Q-Network is responsible for the detection of objects and the association solver is applied to identify the types of objects through confidence score. The tracking and segmentation mechanism is employed at the final stage for identifying movement locations and object segmentation. The experimental validations are performed by using different measures and analysis that showed that the proposed model attained better performances of 98.75% and 95.59% from multiple objects tracking accuracy and dice score respectively.</p>