Self-supervised learning has become a potent paradigm for representation learning in image classification, especially when labeled data is limited. However, current methods often yield suboptimal feature representations and poor clustering, hindering their performance in downstream tasks. To address this, we propose a new self-supervised learning framework that enhances feature embeddings and clustering quality by integrating contrastive learning with cross-entropy-based self-labeling. Our approach employs a two-stage training process: initial pretraining of the backbone network using contrastive learning, followed by further optimization with our hybrid loss function. Extensive experiments on CIFAR, TTSD, and NEU datasets across various evaluation settings demonstrate the effectiveness of our method. Specifically, we achieve accuracies of 84.28% on CIFAR-10, 57.06% on CIFAR-100, and 86.46% on TTSD and 95.00% on NEU datasets for machine vision applications. Beyond classification accuracy, our method also offers a pseudo-labeling mechanism applicable to unlabeled datasets.