<p>The escalating sophistication of cyberattacks, exemplified by Advanced Persistent Threats (APTs), poses an increasingly formidable challenge to the contemporary digital ecosystem. In the face of these escalating threats, advancing research in cybersecurity defense technologies and policies has become essential. This paper focuses on implementing autonomous cyber defense based on reinforcement learning to mitigate the impact of Advanced Persistent Threats. Although numerous researchers have explored autonomous network defense, the computational and analytical capabilities of algorithms often prove inadequate in complex network scenarios, resulting in limited defensive performance. Therefore, the development of robust and efficient reinforcement learning algorithms that are well-suited to network defense environments has become a critical challenge for achieving autonomous network defense. This paper introduces a novel algorithm named Graphsage-PPO (GPPO), which integrates Graph Neural Networks into the Proximal Policy Optimization actor-critic framework. This integration allows the state information of each host to incorporate data from its neighboring hosts, enabling the defense agent to comprehensively consider the complex interdependencies between network hosts. Consequently, the agent can promptly identify potential APT attack paths in complex network scenarios and formulate more precise and effective defensive policies. Extensive experiments conducted in various network attack-defense scenarios, built on the high-fidelity CybORG simulation platform, demonstrate that the GPPO algorithm outperforms several existing algorithms in robustness and performance.</p>