This paper illustrates the application of inertial sensors for acquiring and recognizing patterns in Libras (Brazilian Sign Language). The data acquisitions were conducted using the accelerometer embedded in the Arduino Nano 33 IoT microcontroller, which was affixed to a 3D-printed bracelet placed on the right hand of the volunteers. Integration with the Edge Impulse platform facilitated data storage, processing, and gesture classification. The selected Libras’ signs predominantly utilized movements of the right hand and encompassed everyday gestures such as greetings, expressions of gratitude, personal identification, and other common social interactions. Two distinct approaches were employed for data acquisition and gesture classification using machine learning models. The first method, implemented with a larger pool of volunteers and a smaller set of gestures, achieved higher accuracy when compared to the second method, which had fewer volunteers and more gestures. The first approach achieved 91.4% accuracy, and the second approach 70.4%. A comparative analysis of both training models highlighted the impact of the data acquisition approach on the efficacy of the recognition model. Finally, this research contributes to the advancement of assistive technology. It promotes social inclusion by providing an innovative solution to enhance communication and interaction between Libras speakers and non-Libras speakers.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluation of Inertial Sensors for Sign Language Recognition in Embedded Application Based on Edge Impulse Platform

  • L. S. V. Bôas,
  • G. G. Silva,
  • L. D. L. C. Faxina,
  • A. T. P. Inafuco,
  • T. S. Dias,
  • J. J. A. Mendes Junior

摘要

This paper illustrates the application of inertial sensors for acquiring and recognizing patterns in Libras (Brazilian Sign Language). The data acquisitions were conducted using the accelerometer embedded in the Arduino Nano 33 IoT microcontroller, which was affixed to a 3D-printed bracelet placed on the right hand of the volunteers. Integration with the Edge Impulse platform facilitated data storage, processing, and gesture classification. The selected Libras’ signs predominantly utilized movements of the right hand and encompassed everyday gestures such as greetings, expressions of gratitude, personal identification, and other common social interactions. Two distinct approaches were employed for data acquisition and gesture classification using machine learning models. The first method, implemented with a larger pool of volunteers and a smaller set of gestures, achieved higher accuracy when compared to the second method, which had fewer volunteers and more gestures. The first approach achieved 91.4% accuracy, and the second approach 70.4%. A comparative analysis of both training models highlighted the impact of the data acquisition approach on the efficacy of the recognition model. Finally, this research contributes to the advancement of assistive technology. It promotes social inclusion by providing an innovative solution to enhance communication and interaction between Libras speakers and non-Libras speakers.