<p>Even though autonomous driving has emerged as a prominent study topic, the conventional control systems for autonomous driving are often rule-based and need to be more adaptable to the flow and conditions of traffic that change over time. Recurrent deterministic policy gradient (RDPG) is a strategy for building autonomous driving control systems. Its performance has been shown to be better than some other methods. Consequently, in this study, we make use of the RDPG algorithm to implement our control strategies as well and further give more comprehensive considerations to the learning procedure to obtain better control performance in the testing procedure, e.g., various punishments to avoid vehicle collisions, different speed limitations to avoid slow-driving or fast-driving, distinct rewards to encourage the ego-vehicle to reach the destination, and so on. On the other hand, we also improve the training performance by focusing solely on the critical events during the training procedure. Namely, our training architecture is more efficient based on the same training time (training steps). The road scene and vehicular simulator, AirSim, has been selected as the experimental platform. The findings indicate that our design achieves more accurate and steady outcomes in control and faster convergence in learning compared to an existing RDPG control strategy for autonomous driving in the literature.</p>