Stable and natural locomotion of bipedal robots in complex terrains provides the foundation for their application in real-world scenarios such as rescue and exploration. Robots trained using traditional deep reinforcement learning methods, for instance Proximal Policy Optimization (PPO), often exhibit high robustness owing to the use of standard network structures and conventional reward functions; however, the naturalness of the gaits generated within complex terrain scenarios is typically suboptimal. To this end, this paper proposes an enhanced PPO-Adversarial Motion Priors (AMP) framework specifically tailored for complex terrains. This framework enhances the system’s learning capability by refining the network structures of the PPO policy and value networks, as well as the AMP discriminator. It integrates terrain height detectors and contact force sensors, thereby expanding the observation space to augment the system’s environmental perception capabilities. Concurrently, this paper designs a novel reward function and modifies the self-collision condition from a traditional penalty term to an episode termination condition, aiming to encourage safer behaviors. During training, PPO-Clip and KL divergence constraints are incorporated to prevent excessive magnitudes in policy optimization steps. Experimental results based on the Isaac Lab simulation platform demonstrate that a model employing a deep network structure (six layers, approx. 2.79 million parameters) exhibits exceptional walking robustness and highly natural gaits closely approximating human locomotion across various complex terrains, even when the task reward comprises solely a survival reward with a weight accounting for merely 0.2. In contrast, robots trained using traditional PPO alone typically display only distinctly mechanical gaits.