In an environment where spoken language is the primary means of communication, those who are nonverbal encounter considerable difficulties in expressing themselves and establishing social connections. Our study aims to develop a real-time interactive system that uses technology to recognize Indian Sign Language (ISL) gestures. A dataset containing 2100 ISL gesture videos was used. 100,000 sentences were used to train the grammar error correction model. This system can recognize a subset of ISL gestures, specifically health domain, form meaningful sentences, and translate it to regional languages like Kannada to enable meaningful and seamless communication between the general public and those who are hard of hearing. To accomplish this, computer vision techniques and NLP were used. This paper uses MediaPipe for feature extraction, and Bi-LSTM model is trained on a custom dataset for gesture recognition. The recognized gestures are sent to the fine-tuned transformer (T5) model which generates a grammatically correct sentence. This sentence is forwarded to the Google Translate API which produces the text and speech output in regional language (Kannada). The Bi-LSTM model resulted in a testing accuracy of 99.68%. The BLEU score of the T5 model used for sentence formation is 0.654.