<p>Autonomous exploration in unknown environments requires a delicate balance between exploration and exploitation. This paper introduces SAC-alpha, an enhanced Soft Actor-Critic algorithm with dynamic entropy adjustment, designed to optimize exploration efficiency and adaptability. Integrated into a hierarchical navigation framework, SAC-alpha combines global waypoint selection with local decision-making using deep reinforcement learning. Experimental results in procedurally generated environments demonstrate SAC-alpha’s superior exploration performance, reduced variance, and enhanced adaptability compared to baseline algorithms such as PPO and standard SAC. Despite challenges such as incomplete convergence and computational overhead, SAC-alpha shows promise for complex exploration tasks. Future work will focus on improving sample efficiency, scalability, and real-world deployment.</p>