UAVs (unmanned aerial vehicle) are highly flexible in urban and other scenarios and have been widely used as relay nodes for wireless communications in recent years. However, limited by the batteries that UAVs can carry, maximizing energy efficiency in D2D (Device-to-Device) communications becomes a critical problem UAV-based communications. Existing research focuses on communication between ground-based communication networks and UAVs, with less attention paid to energy efficiency optimization of air-ground networking communication. In this document, an enhanced version of the DA-TD3 algorithm that aims to maximize the overall performance of air-to-ground networks by combining UAV relay and D2D communication. By designing a fair incentive mechanism and training sequence, we can address the out-of-band interference caused by band multiplexing in cellular networks in D2D communication. This will help achieve optimal power control, reduce communication interference, improve the probability of successful D2D link establishment, and significantly enhance the network’s energy efficiency. Then simulation results are carried out to approved our proposed algorithm is effective and reliable.