Development and Prototyping of a Speech-To-Text (STT) Model and Lip Recognition Software for Tiny Machine Learning (TinyML) Systems
摘要
Across the world, there are an increasingly large number of individuals who have hearing impairments. This pressing issue calls for a need to help those who suffer from hearing loss which is why a way for them to understand what others are saying is necessary. To address this issue, a Speech-To-Text (STT) and lip recognition model was developed for Tiny Machine Learning (TinyML) systems. The STT will convert the spoken words into text which can be displayed in real-time at high efficiency and accuracy, while the lip recognition model will search for a lip and identify it in real-time with accurate results. These models are created with Edge Impulse and work hand in hand by first detecting the presence of a lip and then converting the person’s speech into text. These models can be applied to various devices due to their low power consumption. These models are simple designs that can be worked on to create a model that can effectively break the communication barrier between those in and out of the hearing-impaired community.