<p>Rapid urbanization and the surge in vehicle ownership have exacerbated traffic congestion, posing substantial economic, environmental, and social challenges. Traditional traffic signal control methods often struggle to address the dynamic complexities of modern urban traffic, frequently resulting in operational inefficiencies. Reinforcement Learning (RL), with its inherent capacity for real-time learning and adaptation, has emerged as a promising paradigm for optimizing Traffic Signal Control (TSC). RL approaches are particularly well-suited for handling complex traffic states and coordinating global optimization across multiple intersections. Despite notable progress, RL-based systems continue to face significant hurdles, including high computational costs, extensive data requirements, and issues regarding generalizability across diverse traffic scenarios. This paper synthesizes current RL-based models for TSC and highlights recent advancements in the field. It provides a comprehensive review of prominent approaches, categorizes existing studies based on their methodological frameworks, and conducts a technical evaluation of classical RL-based methods to assess their performance across varied traffic conditions. Finally, the remaining challenges and potential future directions for RL-based TSC are critically examined.</p>