Machine learning methods involve both supervised and unsupervised learning. On the other hand, semi-supervised and self-supervised learning also play a crucial role, with many applications increasingly relying on these paradigms. Real-world applications such as healthcare, marketing, and digital media generate massive data. However, labeling this data is often impractical due to resource constraints. Consequently, there is an increasing need for techniques that can efficiently utilize limited labeled data to facilitate the labeling of large amounts of unlabeled data. Semi-Supervised Clustering (SSC) addresses this challenge by using available labeled data to guide the clustering process. However, maintaining clustering quality and handling outliers remain significant challenges. To tackle these issues, we proposed Radial Expansion Compact Clusters (RECC), a novel framework based on the label-based clustering paradigm to form quality compact clusters while efficiently assigning labels to unlabeled data. RECC also incorporates an outlier detection mechanism to minimize the impact of misleading data on clustering process. Extensive experiments on real-world datasets demonstrate that the proposed RECC method outperforms the conventional methods in improving clustering quality.