Introduction. Distinguishing between subtypes of contact dermatitis poses significant diagnostic challenges due to their subtle and overlapping clinical manifestations. The development of Artificial Intelligence (AI) models to differentiate these conditions holds promise for enhancing diagnostic accuracy. This study aimed to enhance the F1 score and other evaluative metrics of an AI model for the classification of atopic dermatitis and contact dermatitis images, utilizing improved data labeling and annotation methodologies. Methodology. We compiled a diverse dataset of images representing various skin conditions, with an emphasis on contact dermatitis, which was partially labeled by three dermatologists. Subsequently, we incorporated automated annotations, guided by dermatological expertise, to ensure the accuracy and detail of these annotations. This process involved the precise delineation of affected areas and a thorough description of clinical features. A deep learning model was then trained on this augmented dataset. We assessed the model's performance using precision, recall, and the F1 score, comparing these metrics against those achieved by models trained on datasets with less rigorous annotations. Results. The implementation of advanced labeling and annotation significantly enhanced the model’s performance. The improvements were evident in the precision and sensitivity, which averaged 83% (Confidence Interval [CI] 80–85%) and 86% (CI 84–87%), respectively. The average F1 score increased from 0.62 to 0.83 across 10 iterations of model training. The mean labeling concordance of individual experts relative to the reference standard was 85.4%  ±  2.0 (SEM), while automated AI had an 87.9% concordance rate with the reference standard, which was not significantly different from the experts’ rate (P  =  0.15). Conclusion and Discussion. This study highlights the essential role of high-quality data annotation in developing AI models for diagnosing dermatological conditions. By incorporating detailed, expert-driven annotations, we significantly improved the accuracy of an AI model for differentiating contact dermatitis subtypes.