Kontho: An AI-Driven Smart Glove for Enhanced Sign Language Communication
摘要
In India, over 63 million individuals are affected by hearing loss, yet less than 350 certified sign language interpreters are available, thus leading to communication barriers and significant social and economic exclusion. This paper introduces Kontho, an ergonomic, AI-driven, multi-sensor glove integrated with a mobile app for real-time sign language communication via American Sign Language (ASL). Utilizing flex sensors, accelerometers, gyroscopes and EMG sensors, Kontho captures gestures and muscle activity which is processed through a combination of ensemble models and Large Language Models (LLMs) to enhance communication by predicting and completing incomplete gestures. Evaluated on a dataset of 23 ASL gestures at over 3867 data points with an accuracy of 99.61%, Kontho offers a scalable solution to foster social inclusion and address the economic isolation of hearing and speech-impaired individuals, enabling their active participation in the society.