<p>Accurate decision-making within highly interactive driving environments is vital for the safety of self-driving vehicles. Despite the significant progress achieved by the existing models for autonomous vehicle decision-making tasks, there remains untapped potential for further exploration in this field. Previous models have focused primarily on specific scenarios or single tasks, with inefficient sample utilization and weak robustness problems, making them challenging to apply in practice. Motivated by this, a robust decision-making method named DRL-EPKG is proposed, which enables the simultaneous determination of vertical and horizontal behaviors of driverless vehicles without being limited to specific driving scenarios. Specifically, the DRL-EPKG integrates human driving knowledge into a framework of soft actor-critic (SAC), where we derive expert policy by a generative model: variational autoencoders (VAE), train agent policy by employing the SAC algorithm and further guide the behaviors of the agent by regulating the Wasserstein distance between the two policies. Moreover, a multidimensional reward function is designed to comprehensively consider safety, driving velocity, energy efficiency, and passenger comfort. Finally, several baseline models are employed for comparative evaluation in three highly dynamic driving scenarios. The findings demonstrate that the proposed model outperforms the baselines regarding the success rate, highlighting the practical applicability and robustness of DRL-EPKG in addressing complex, real-world problems in autonomous driving.</p>