This project develops a system that transforms written or spoken words into sign language animations, specifically Macedonian Sign words (MSL), to help individuals who are hard of hearing communicate. A virtual interpreter that transforms spoken language into animated sign language is developed using 3D visualization, computer vision, and machine learning. Although there are only approximately thirty MSL interpreters available in Macedonia for the deaf community, the system utilizes pose estimation technologies, such as Blaze Pose, to extract body keypoints from 2D video inputs. This makes it possible for the creation of precise 3D sign language movements. These keypoints are mapped to a skeleton with specialized software, and the skeleton data is then used to generate realistic, dynamic sign language animations for 3D models. Techniques like interpolation and inverse kinematics are used to generate and modify the animations in order to guarantee realistic movements and movement. This project could be expanded to include other sign languages around the world, improving accessibility in a variety of domains such as education, healthcare, and public services. It improves social inclusion and equal opportunity for the deaf and hard-of-hearing community while also revolutionizing sign language education. The goal is to develop a scalable solution that may be expanded to support more sign languages, so contributing to more inclusive communities globally.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automated 3D Sign Language Animation Using Machine Learning Algorithms

  • Teodora Kochovska,
  • Bojana Velichkovska,
  • Marija Kalendar

摘要

This project develops a system that transforms written or spoken words into sign language animations, specifically Macedonian Sign words (MSL), to help individuals who are hard of hearing communicate. A virtual interpreter that transforms spoken language into animated sign language is developed using 3D visualization, computer vision, and machine learning. Although there are only approximately thirty MSL interpreters available in Macedonia for the deaf community, the system utilizes pose estimation technologies, such as Blaze Pose, to extract body keypoints from 2D video inputs. This makes it possible for the creation of precise 3D sign language movements. These keypoints are mapped to a skeleton with specialized software, and the skeleton data is then used to generate realistic, dynamic sign language animations for 3D models. Techniques like interpolation and inverse kinematics are used to generate and modify the animations in order to guarantee realistic movements and movement. This project could be expanded to include other sign languages around the world, improving accessibility in a variety of domains such as education, healthcare, and public services. It improves social inclusion and equal opportunity for the deaf and hard-of-hearing community while also revolutionizing sign language education. The goal is to develop a scalable solution that may be expanded to support more sign languages, so contributing to more inclusive communities globally.