Skin color naturally varies among individuals due to genetic differences, but facial discoloration can signal cosmetic or medical issues. Although skin tone is primarily genetic, discoloration can occur due to various factors, including prolonged sun exposure, cosmetic issues, and medical conditions that may require attention. This study aims to assess the severity of facial discoloration across different skin types. To achieve this, 6,042 images were collected and augmented to enhance model robustness in analyzing facial discoloration severity. A cosine similarity measure was employed for identification based on the Fitzpatrick scale. Gaussian blur filtering was applied to improve structural retention across skin tones and preserve image quality. Canny edge detection was employed to segment discoloration regions, which were subsequently used to distinguish between images with and without discoloration automatically. Three machine learning models were tested, with results showing that the Random Forest algorithm achieved the highest accuracy (75%), outperforming Support Vector Machine (SVM) and MobileNetV2. A Convolutional Neural Network (CNN) was then applied for facial discoloration detection, achieving an accuracy of 76%. Moreover, a rule-based approach was used to measure facial discoloration by calculating the pixel area of the image and was tested on five (5) images. However, due to variability in skin types and real-world conditions, the model’s performance remains a challenge, requiring further optimization to enhance robustness and generalization.