This paper investigates a distinctive spacecraft pursuit-evasion game utilizing impulse thrust. In this scenario, a target spacecraft, carrying critical assets for a pivotal mission, is constrained to a predefined orbit without maneuvering. Several accompanying spacecraft orbit nearby to provide protection. The pursuer aims to successfully attack the target while evading detection by the accompanying spacecraft. The transfer process for the pursuer involves a combinatorial optimization challenge, requiring optimal time-fuel control and an online decision algorithm for swift responsiveness. To tackle this optimization problem, we propose a reinforcement learning (RL)-based method. This study considers practical engineering constraints on impulse magnitude, engine shutdown time, and sensor measurement noise limits, increasing the problem’s complexity. Numerical examples demonstrate the effectiveness of our approach.