In clustering problems, finite mixture models (FMMs) are frequently used due to their tractability through maximum likelihood estimation. Although the expectation-maximization (EM) algorithm assists in fitting these models, it is vulnerable to local optima and distortions from both mild and gross outliers. We investigate the use of the self-paced learning (SPL) algorithm for a more robust approach. The SPL algorithm estimates the parameters of a FMM by introducing observations in a determined order, utilizing weighted terms of a penalized likelihood function. We found the SPL algorithm does provide resilience toward highly contaminated data but estimates a significant number of degenerate solutions. The proposed novel component SPL algorithm determines the order in which observations are introduced at a component level. The performance of the proposed algorithm is evaluated both through simulated and real-world applied data sets. Results show a reduction in estimation bias for highly contaminated data, with an overall reduction in the number of degenerate solutions.