In recent years, one of the key innovations in developing 5G networks to enhance the worldwide connectivity is initialized by utilizing satellite and terrestrial networks as they have high capacity and extensive coverage. Even though, still there are challenges in these systems such as resource limitations, variable channel conditions, and optimal power allocation. Hence, this research proposes a robust model namely Double Deep Q-Network (DDQN) to maintain reliable connectivity, improve Quality of Service (QoS) and attain optimal power allocation. Specifically, DDQN assists for optimal power allocations due to its ability to reduce overestimation bias in action evaluation by using two distinct Q-networks for action selection and estimation. The system model is designed by incorporating Low Earth Orbit (LEO) satellite networks, base station, and channel modeling. From the results, the proposed DDQN achieved outstanding results in terms of sum throughput along with user requirements including K = 20 (179.5 Mbps), K = 30 (230.4 Mbps), K = 40 (283.2 Mbps), and K = 50 (327.0 Mbps), respectively, when compared with existing Satellite Heterogeneous Graph Neural Network (SHGNN).