Neonatal Jaundice is a common condition that, if left undiagnosed or untreated, can lead to severe complications, including kernicterus and neurological damage. Most of the full term and preterm infants develop the neonatal Jaundice. Jaundice mainly occurs due to the increased levels of serum bilirubin, principally due to the disintegration of the red blood cells. Generally the term babies have ’physiological’ Jaundice, which is treated by responding to a very small span period of phototherapy and requires no other treatment. Some of the babies have increasing bilirubin levels eventually leading to the risk of kernicterus. Any infant with a high serum bilirubin or a swiftly rising bilirubin level should be given immediate attention to avoid neuro-toxicity. Thus phototherapy treatment are used for the infant with high levels of Bilirubin. Deep learning is the state-of-the-art technologies, having powerful computing capabilities to harness the features within an image due to its complex computing structure. This research proposes an intelligent system for the early detection of neonatal Jaundice using machine learning and image pre-processing techniques such as segmentation, feature detection, and Classification. The system leverages digital image analysis to assess skin color variations and a Sequential deep learning model for classification. The current approach aims at providing a painless, non-invasive, cost-effective, and swift diagnosing and screening solution. Experimental test results demonstrate the efficiency and reliability of the system compared to conventional diagnostic methods. The proposed model aims to classify a particular image of the infant as either Jaundice or Normal. The said model generates an accuracy of 99.58%.