Palm leaf has been one of the oldest writing media around the world. Palm leaf manuscripts, known as lontar, have long been used as a medium for recording and transmitting knowledge in Balinese culture in Indonesia. These intricately crafted manuscripts have been used by the ancestors to pass their knowledge on various subjects such as environment management, survival techniques, medication, astronomy, and more. These manuscripts also contain valuable information about ancient Balinese life encompassing religion, culture, art, and various aspects of local wisdom. However, due to their organic nature, they deteriorate over time making it extremely difficult to handle and recognize the text in those documents. The proposed model in this paper aims to develop a system for detecting and recognizing palm leaf Balinese text using deep neural networks. The recognition of the text is especially a challenging task due to the complex nature of handwritten characters and variations in text styles. The proposed model utilizes the CRAFT (Character Region Awareness for Text Detection) model’s ability to work on large and small texts for text detection in palm leaf manuscripts, and VGGNet (Visual Geometry Group Network) a very deep Convolution Neural Network (CNN) for text recognition The proposed system provides an accurate and efficient solution for palm leaf Balinese text detection and recognition.