To enable autonomous decision-making in multi-Unmanned Aerial Vehicle (UAV) air combat, this paper proposes the Artificial Potential Field-Cooperative-Multi-Agent Proximal Policy Optimization (APF-C-MAPPO) strategy generation algorithm, integrating game theory, heuristic methods, and neural networks. Firstly, after building the model for multi-UAV combat scenario, a dynamic cooperative reward mechanism is designed based on pareto optimality, where team rewards are adjusted via a pareto reward coefficient. Then, the APF method maps close-range strike tasks into the air combat field, guiding UAVs to proactively engage enemy targets during early training and accelerating strategy convergence. Finally, a multi-agent reinforcement learning framework based on MAPPO is constructed, combining Gated Recurrent Unit (GRU) networks, Batch Advantage Normalization (BAN) advantage estimation, and a shared experience buffer, with strategy optimization achieved via parallel self-play training. Simulation results demonstrate that the proposed method enables efficient policy learning and intelligent cooperation among UAV teams in complex games, achieving higher win rates and lower training times.