Quadcopters offer remarkable agility and versatility, yet their nonlinear dynamics, actuator constraints, and sensor noise complicate precise control. Conventional cascade controllers, reliant on domain expertise and extensive parameter tuning, struggle with adaptability to hardware variations and time-consuming setup, limiting their effectiveness for autonomous flight. Motivated by these challenges, this study proposes an integrated deep reinforcement learning (DRL)-based control framework for robust trajectory tracking of quadrotors, eliminating the need for manual gain adjustments. A novel actor-critic architecture directly maps quadrotor states to motor commands, trained in a dynamics-based simulator to optimize the control policy efficiently. The trained policy is deployed on-board a Crazyflie Bolt quadrotor, equipped with brushless motors and electronic speed controllers (ESCs), without requiring fine-tuning. Experimental results demonstrate accurate tracking of complex trajectories, underscoring the method’s data efficiency, scalability, and ability to bridge the simulation-to-reality gap. This approach outperforms traditional methods, offering a versatile, high-performance control solution for diverse quadrotor platforms and advancing autonomous aerial navigation.