One of the urgent problems in traffic is to reduce congestion and waiting time of vehicles participating in traffic. This paper focuses on the employment of reinforcement learning algorithms such as Double Deep Q-Network and Proximal Policy Optimization to reduce vehicle waiting times in three-way and four-way intersections with heterogeneous traffic conditions. At the same time, we also investigate the performance of the algorithm in both heavy and clear traffic scenarios. Our method dramatically lowers waiting times and improves traffic flow efficiency, according to extensive testing. Significantly, the outcomes reveal remarkable performance gains in both three-way and four-way intersection, indicating the possibility of widespread use in urban traffic control. When compared to the fixed time strategy, the experimental results likewise perform better in both clear and congested traffic scenarios. This study demonstrates the effectiveness of intelligent traffic control systems in varied and complex traffic environments.