Higgs boson searches at the Large Hadron Collider require event selection methods that maximise discovery significance while maintaining reliable performance across varying data conditions. Machine learning models trained to optimise significance-based metrics can produce unstable decision thresholds and inconsistent results across validation splits, potentially affecting the robustness of experimental conclusions. This study investigates whether supervised contrastive representation learning can improve the stability of classification in significance-driven analyses using the HiggsML benchmark dataset of simulated \(H \rightarrow \tau ^{+}\tau ^{-}\) events. The proposed framework integrates supervised contrastive pre-training with a hybrid parallel ensemble combining an FT-Transformer and XGBoost classifier. Evaluation using repeated \(5 \times 5\)-fold cross-validation shows reduced fold-to-fold variability in Approximate Median Significance (AMS) compared with conventional focal loss training, while maintaining competitive performance (AMS = 3.74 on the full dataset). The findings indicate that improved feature representations can lead to more consistent model behaviour and more reliable threshold selection under global significance objectives. This study systematically investigates the role of supervised contrastive representation learning in improving AMS stability and threshold consistency within a severely imbalanced tabular classification setting. These advances align with the objectives of sustainable technological innovation and resilient infrastructure outlined in United Nations Sustainable Development Goal 9 (Industry, Innovation and Infrastructure).
