In this paper, we investigate the application of a reinforcement learning algorithm known as the Dueling Double Deep Q-Network to discover bus scheduling strategies and compare them against conventional approaches. In particular, we look into real-time control strategies where buses may choose to stay or leave at bus stops. We explore both waiting time and travel time as the optimization objectives. The results for uniform bus frequency show that average waiting time can be reduced by allowing buses to stay longer at stops with higher passengers’ arrival rate but at the cost of increased average travel time. This is also supported by our analytical calculation on a theoretical bus loop model. We then apply our method to a model based on a real world bus loop in Nanyang Technological University. The results highlight the potential benefit of reinforcement learning methods to find novel strategies that can be better than conventional approaches.