Sign language is an indispensable communication tool that gives them a way to convey ideas and emotions of deaf and dumb people. However, this presents challenges in terms of inclusion, as an interpreter is often needed to translate sign language into spoken or written language. This study focuses on developing a sign language recognition and Telugu-to-text conversion system to improve communication and accessibility for hearing-impaired Telugu speakers. The system uses machine learning techniques for recognizing and converting gestures into Telugu text. Using cameras to record the user’s sign language movements, the system processes the data and translates it into Telugu letters. Proposed methodology gains an accuracy of 74.65%, a recall of 74.65%, and a precision of 76.61%. The novelty of our methodology lies in its focus on Telugu Sign Language (TSL), an area with limited research and resources. While many studies have addressed sign languages like ASL (American Sign Language) or BSL (British Sign Language), our research fills a critical gap by developing a robust system specifically for TSL.