Across the world, there are an increasingly large number of individuals who have hearing impairments. This pressing issue calls for a need to help those who suffer from hearing loss which is why a way for them to understand what others are saying is necessary. To address this issue, a Speech-To-Text (STT) and lip recognition model was developed for Tiny Machine Learning (TinyML) systems. The STT will convert the spoken words into text which can be displayed in real-time at high efficiency and accuracy, while the lip recognition model will search for a lip and identify it in real-time with accurate results. These models are created with Edge Impulse and work hand in hand by first detecting the presence of a lip and then converting the person’s speech into text. These models can be applied to various devices due to their low power consumption. These models are simple designs that can be worked on to create a model that can effectively break the communication barrier between those in and out of the hearing-impaired community.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Development and Prototyping of a Speech-To-Text (STT) Model and Lip Recognition Software for Tiny Machine Learning (TinyML) Systems

  • Logan Wong Duran,
  • Mithil Darur,
  • Kok Zuea Tang

摘要

Across the world, there are an increasingly large number of individuals who have hearing impairments. This pressing issue calls for a need to help those who suffer from hearing loss which is why a way for them to understand what others are saying is necessary. To address this issue, a Speech-To-Text (STT) and lip recognition model was developed for Tiny Machine Learning (TinyML) systems. The STT will convert the spoken words into text which can be displayed in real-time at high efficiency and accuracy, while the lip recognition model will search for a lip and identify it in real-time with accurate results. These models are created with Edge Impulse and work hand in hand by first detecting the presence of a lip and then converting the person’s speech into text. These models can be applied to various devices due to their low power consumption. These models are simple designs that can be worked on to create a model that can effectively break the communication barrier between those in and out of the hearing-impaired community.