The article proposes a new combined approach to investigation the problem of classification on real-world noisy dataset using a multi-stage semi-supervised learning method. The main idea of this approach is based on combining two methods: self-supervised learning on unlabeled data using Contrastive Loss Nets and semi-supervised learning label propagation using an enhanced Poisson Seidel learning technique. The proposed approach offers significant advantages, as it allows for preliminary classification without labels, strengthening the distinctions between classes, and then using a minimal amount of labeled data for final classification. This is demonstrated through the analysis of synthetic data from different intersecting “Two moons” and real medical dataset on heart disease—“Cardio Vascular” Accuracy in the first case exceeds 82%, and for the second example—73%, which is one of the best result on the Kaggle database when compared to any other known 20 methods.