Customizable AAC Device: ML-Enhanced Speech Aid for Impairment
摘要
Individuals with speech impairments often face significant communication challenges, requiring innovative solutions. With over 300+ sign languages globally, only about 3% are widely recognized, such as American, British, and Indian Sign Language. This research introduces a pioneering Augmentative and Alternative Communication (AAC) device designed for easy wearability around the neck, integrating a camera, Raspberry Pi, and Google’s Text-to-Speech technology. This device enables users to create and utilize their own sign languages, enhancing communication with family and the broader community without needing to learn standardized sign languages. The device is complemented by a user-friendly mobile application that simplifies device management and empowers users to create and customize their own sign languages. Additionally, it contributes to filling a significant research gap by generating a unique dataset of first-person perspective sign language images, supporting more accurate and personalized communication solutions. The effectiveness of this device has been validated using machine learning models, including a random forest classifier and a convolutional neural network (CNN), achieving high accuracy levels (84–99%) in translating personalized sign language gestures into speech. This approach marks a substantial advancement over existing methods, offering a novel communication aid for individuals with speech impairments.