Class Incremental Learning (CIL) has attracted a lot of attention due to its ability to continuously acquire knowledge from streaming data. However, catastrophic forgetting remains a central challenge in CIL. To alleviate this issue, we propose using Controlled Cluster Separation-Gaussian Mixture Model (CCS-GMM) to preserve knowledge of different class. By representing each class with multiple clusters, CCS builds more robust feature distributions that are less affected by new classes, thereby mitigating forgetting. Meanwhile, GMM can effectively capture the distributions of clusters, thereby facilitating a more faithful description of each class. Extensive experiments demonstrate that our method improves performance across several CIL benchmarks.