Knowledge graphs (KGs) in real-world applications are inherently dynamic and continuously evolving. Continual Knowledge Graph Embedding (CKGE) aims to integrate new knowledge while preserving previously acquired knowledge, which ensures an optimal balance between learning and retention. However, existing CKGE methods predominantly focus on mitigating catastrophic forgetting but neglect the continuous performance improvement from the efficient integration of new knowledge. In this work, we propose a novel model based on Contrastive Masked KG Autoencoder with Joint Anti-Forgetting module, termed CMJA. It facilitates the integration of new knowledge and strengthens the retention of previous knowledge. To effectively acquire new knowledge, CMJA incorporates a contrastive masked KG autoencoder, which improves the ability of the embeddings to capture distinguishing features. Additionally, to consolidate the retention of previously learned knowledge, we introduce a joint anti-forgetting module, which imposes stricter control over minor discrepancies and enhances the model’s robustness to noise. It mitigates catastrophic forgetting more comprehensively. Experimental results show that CMJA achieves superior performance in the link prediction task. Subsequent experiments show that CMJA achieves a more desirable balance between plasticity and stability.