To address the challenge of balancing local decision-making and global coordination in a multi-ship-cooperative-control framework tasks, this paper proposes a hierarchical reward mechanism that divides rewards into three levels: basic, cooperative, and adaptive. The basic level guides navigation tracking and obstacle avoidance; the cooperative level maintains formation and relative positions; the adaptive level dynamically adjusts weights according to environmental complexity, achieving multi-scale refined incentives. Experimental results demonstrate that the proposed approach significantly improves formation stability and task efficiency in complex sea conditions.