We propose a novel optimization model for partitional clustering with unknown number of clusters, combining the Minimum of Sum of Squared Clustering (MSSC) as the loss term and the \(\ell _{2}\) grouping centers as the regularization term. This combination leads to a nonconvex, nonsmooth optimization problem, for which we investigate efficient techniques based on DC (Difference of Convex functions) programming and DC Algorithm (DCA). Using a suitable DC decomposition we derive a simple and efficient DCA scheme which requires only matrix–vector products at each iteration. The proposed DCA starts with a very large number of clusters and provides as output an optimal partition along with the optimal number of clusters. Numerical experiments on various synthetic and real-world datasets demonstrate the efficiency of our algorithm in terms of both clustering quality and convergence speed. They outperform four reference algorithms, including k-means variants (X-means, G-means), penalized regression approaches (PRclust), and the Gaussian mixture model (mclust), not only in terms of quality but also on scalability. Notably, it realizes an excellent trade-off between solution quality and running time, making it a good choice for practical clustering tasks with Big data.