In multi-air-to-air missile cooperative strike missions, traditional analytical guidance laws often fall short when facing complex time coordination constraints. To address this limitation, this paper proposes a predefined arrival time difference cooperative guidance law based on deep reinforcement learning. By fully exploiting the feature representation capability and policy adaptability of deep neural networks, the proposed method enables precise temporal control for simultaneous missile launch and staggered-time impact. In the design and training of the guidance law, a three-dimensional relative motion model between missile and target is first established. Based on this, a master–slave cooperative guidance strategy is developed, in which the master missile incorporates a time bias term generated by reinforcement learning into a traditional proportional navigation guidance law, while the slave missile follows a standard PNG. An improved Proximal Policy Optimization framework is constructed for training the master missile’s policy. The state space includes features such as the flight states of both missiles and line-of-sight information, while the reward function jointly considers terminal hit accuracy and the predefined arrival time difference constraint. Simulation results under multiple scenarios demonstrate that the proposed guidance strategy can achieve accurate terminal impact while satisfying the required arrival time difference between missiles, thereby significantly enhancing cooperative strike effectiveness.