This research introduces a deep reinforcement learning (DRL) system that integrates adaptive experience replay to optimize beamforming and power control in 5G networks. The novel approach focuses on improving sample efficiency speeding up convergence and enhancing the signal to interference plus noise ratio (SINR) and sum rate capacity. Notable contributions include the use of an adaptive experience replay (AER) buffer that dynamically adjusts the importance of experiences based on their learning impact enhancements to the Deep Q Network (DQN) architecture and thorough simulations to assess performance across various 5G network scenarios. Simulation outcomes reveal enhancements in network performance and computational efficiency when compared to conventional methods and current DRL algorithms. The proposed AER-DQN model achieves better SINR and sum rate capacity while demonstrating quicker convergence and reduced training duration. This study tackles the drawbacks of existing approaches by offering an adaptable solution, for optimizing joint beamforming and power control in 5G networks.