In recent years, the technology for autonomous exploration by UAVs has developed rapidly, leading to the emergence of various methods. However, most of these methods assume drift-free localization, which is impossible to achieve in real environments. This results in poor map reconstruction quality and can even affect the safety of UAV flight. In this work, we propose a systematic exploration and loop closure planning framework that ensures exploration efficiency while minimizing the impact of localization drift, thereby achieving better map reconstruction results. We propose a loop closure strategy based on deep reinforcement learning, which can actively perform loop closures to correct localization errors in cases of severe drift, ensuring both mapping quality and flight safety. Extensive experiments in simulations have demonstrated the effectiveness of the proposed system and strategy.