Communication is one of civilization’s fundamental demands. About 20% of the global population has disabling hearing loss. The deaf and hard-of-hearing people mostly use sign language. It may also be used by individuals who can listen but cannot speak and by those who have deaf family members. Our proposed method aims to convert real-time data of Indian Sign Language (ISL) into text and audio data. The equivalent American Sign Language (ASL) field has seen many studies. However, some obstacles hinder further analysis of ISL. The main obstacle preventing further ISL research has been the lack of standard datasets, hidden characteristics, and local language variance. The majority of the work is based on handcrafted facets. Indian Sign Language (ISL) has static and dynamic signs and various gestures for the same expression depending on the area of India. It makes constructing a standardized dataset very challenging. Furthermore, there is no standard collection accessible. Hence, we were forced to make our dataset which consists of alphabets (A-Z) and numbers (0–9). The proposed method uses a custom-made Convolution Neural Network (CNN) model to classify 36 signs. Once the CNN model recognizes the sign, the output label is printed as text. This proposed model achieves a validation accuracy of 99.4%.