Communication via Sign Language is important for people who can’t speak and hear. However, ordinary people find it difficult to communicate through sign language with hard-of-hearing people. In this paper, we propose to develop a recognition system that can convert an Indian Sign Language into regular text. The study uses Convolutional Neural Networks for Indian sign language detection and data generators for optimizing memory and CPU utilization. The research uses 42000 images of 35 signs, including A to Z alphabets and 1 to 9 numbers. With this collected dataset of images, we have experimented with CNN architecture by changing the different hyper-parameters in terms of filters and optimizers. A model trained with an SGD optimizer achieved an accuracy of 99.99%. Further, a GUI-based tool is developed to recognize Indian Sign Language hand gestures in real-time and convert the sign language to normal readable language.