<p>Autonomous driving technology is constantly developing to a higher level of complex scenes, and there is a growing&#xa0;demand for the utilization of end-to-end data-driven control. However,&#xa0;the end-to-end path tracking process often encounters challenges in learning efficiency and generalization. To address this issue, this paper designs a deep deterministic policy gradient (DDPG)-based reinforcement learning strategy that integrates imitation learning and feedforward exploration in the path following process. In imitation learning, the path tracking control data generated by the model predictive control (MPC) method is used to train an end-to-end steering control model of a deep neural network. Another feedforward exploration behavior is predicted by road curvature and vehicle speed, and adds it and imitation learning to the DDPG reinforcement learning to obtain decision-making experience and action prediction behavior of the path tracking process. In the reinforcement learning process, imitation learning is used to update the pre-training parameters of the actor network, and a feedforward steering technique with random noise is adopted for strategy exploration. In the reward function, a hierarchical progressive reward form and a constrained objective reward function referring to MPC are designed, and the actor-critic network architecture is determined. Finally, the path tracking performance of the designed method is verified by comparing various training results, simulations, and HIL tests. The results show that the designed method can effectively utilize pre-training and feedforward prior experience to obtain optimal path tracking performance of an autonomous vehicle, and has better generalization ability than other methods.&#xa0;This study provides an efficient control scheme for improving the end-to-end control performance of autonomous vehicles.</p>