<p>Automated microstructure classification has the potential to improve the consistency and scalability of metallurgical characterisation; however, industrial datasets frequently exhibit severe class imbalance, where safety-critical microstructures are substantially underrepresented. This study presents a baseline evaluation of rare-phase detection in ultra-high carbon steel (UHCS) microstructures using deep learning. The dataset comprised 961 scanning electron microscopy (SEM) micrographs spanning seven microstructural classes, with Martensite represented by only 36 images (3.75% of the dataset). A frozen ResNet50 transfer-learning framework was evaluated using a stratified train–test split, online data augmentation, and imbalance-aware learning strategies. The baseline model achieved 52.8% accuracy, with Martensite precision of 0.600, recall of 0.429, and F1-score of 0.500. Class-weighted loss increased Martensite recall to 0.857 but reduced overall accuracy to 29.0%, indicating a substantial increase in false-positive predictions. Focal loss achieved the strongest overall performance, with 56.0% accuracy, 36.5% balanced accuracy, and a Matthews Correlation Coefficient (MCC) of 0.388. To assess robustness, stratified five-fold cross-validation was performed, yielding a macro-F1 score of 0.133 ± 0.009 and MCC of 0.134 ± 0.029, highlighting limited multi-class generalisation under severe class imbalance. Additional analyses indicated that classification performance was influenced by class overlap and dataset characteristics, including magnification variability. Overall, the results demonstrate that meaningful microstructural information can be learned from severely imbalanced UHCS datasets; however, reliable rare-phase classification remains constrained by limited minority-class representation and substantial overlap between classes. The study establishes a reproducible baseline for rare-phase detection in UHCS microstructures and identifies key challenges for future research.</p>