This study introduces an innovative framework designed to enhance communication for deaf individuals through an automatic sign language recognition (SLR) mobile app. The system improves real-time processing accuracy and user experience by employing a smart glove embedded with flex sensors and a gyroscope to accurately capture hand and finger movements. These sensors provide continuous data on finger bending and hand orientation, which are crucial for interpreting sign language gestures. An Arduino microcontroller collects and processes this data, transmitting it to a custom machine learning model for real-time analysis. The machine learning (ML) module compares the sensor input with pre-trained models to identify gestures. Once recognized, the corresponding word is displayed on a smartphone screen and converted into audible speech via the mobile app. This study specifically focuses on detecting emergency-related words such as ‘food’, ‘water’, ‘shelter’, and ‘help/rescue, which are crucial in crisis situations. Preliminary findings indicate that the smart glove system accurately recognizes and translates various sign language gestures using ML model(M-f1 score 0.99), demonstrating its potential as an effective communication tool. This innovation has the potential to greatly enhance the quality of life for deaf individuals by facilitating essential interactions and promoting greater accessibility and inclusivity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

EmSLR Framework: Enhancing Communication for Deaf Individuals During Mass Emergency

  • Saugata Sarkar,
  • Sayan Datta,
  • Soumodeep Banerjee,
  • Dhritiman Mukherjee,
  • Moumita Basu

摘要

This study introduces an innovative framework designed to enhance communication for deaf individuals through an automatic sign language recognition (SLR) mobile app. The system improves real-time processing accuracy and user experience by employing a smart glove embedded with flex sensors and a gyroscope to accurately capture hand and finger movements. These sensors provide continuous data on finger bending and hand orientation, which are crucial for interpreting sign language gestures. An Arduino microcontroller collects and processes this data, transmitting it to a custom machine learning model for real-time analysis. The machine learning (ML) module compares the sensor input with pre-trained models to identify gestures. Once recognized, the corresponding word is displayed on a smartphone screen and converted into audible speech via the mobile app. This study specifically focuses on detecting emergency-related words such as ‘food’, ‘water’, ‘shelter’, and ‘help/rescue, which are crucial in crisis situations. Preliminary findings indicate that the smart glove system accurately recognizes and translates various sign language gestures using ML model(M-f1 score 0.99), demonstrating its potential as an effective communication tool. This innovation has the potential to greatly enhance the quality of life for deaf individuals by facilitating essential interactions and promoting greater accessibility and inclusivity.