IoT Enabled Indian Postal Sign Recognition System: Facilitating Communication Among Individuals with Hearing and Speech Impairments
摘要
Individuals with hearing and speech impairments often face significant communication obstacles, especially in public institutions such as post offices. This paper introduces a transformative solution to alleviate this barrier by utilizing IoT and computer vision techniques to develop a Postal Sign Recognition System tailored explicitly for Indian Sign Language (ISL). Initially, IoT techniques are utilized to collect Indian postal sign language using a Raspberry Pi and a web camera. Subsequently, computer vision techniques, including preprocessing, object tracking, and recognition, are applied to the data collected. The preprocessing methods, such as frame differencing and contour analysis, are employed to eliminate background noise from the captured video frames. The optical flow method is then used to localize gestures within the video stream accurately. In the recognition phase, a 3D convolutional neural network (3DCNN) model interprets gestures, translating them into textual output displayed on an LCD screen connected to the Raspberry Pi. The system achieves a gesture prediction accuracy of 79.80%. This IoT-based ISL recognition system empowers post office staff to effectively understand and respond to the communication needs of customers with hearing and speech impairments.