In critical situations where vocalizing distress may not be feasible, identifying and responding to silent distress signals can be life-saving. This paper presents a proof-of-concept system that automatically generates alerts by lipreading silent distress signals by using a deep learning model to identify them. The model records lip movements, converts them to text, and looks for distress-related terms like “help” and “please”. Upon identifying these keywords, the system sends an automated email alert to a designated authority, potentially aiding individuals who are unable to call for help. The system’s architecture employs a 3D Conv-RNN model, which integrates 3D convolutional layers for feature extraction and bidirectional LSTM layers to capture temporal dependencies in lip movements, supporting accurate transcription of silent commands. Over 15 epochs, the model showed a consistent decrease in both training loss, from an initial 94.254 to 56.7217, and validation loss, reducing from 72.1666 to 52.6722, reflecting improvements in transcription accuracy as predictions evolved from placeholder outputs to partial correctness. While this small-scale model demonstrates feasibility, it lays the groundwork for advancing non-intrusive, real-time alert systems through deep learning-based lipreading. The novelty of this system lies in its application of lipreading for emergency assistance, bridging a critical gap in silent distress detection technology.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Lipreading for Sending Distress Signals by Using Deep Learning Model

  • Anushka Tonk,
  • V. S. Siddarth,
  • R. Ashoka Rajan,
  • A. Swaminathan

摘要

In critical situations where vocalizing distress may not be feasible, identifying and responding to silent distress signals can be life-saving. This paper presents a proof-of-concept system that automatically generates alerts by lipreading silent distress signals by using a deep learning model to identify them. The model records lip movements, converts them to text, and looks for distress-related terms like “help” and “please”. Upon identifying these keywords, the system sends an automated email alert to a designated authority, potentially aiding individuals who are unable to call for help. The system’s architecture employs a 3D Conv-RNN model, which integrates 3D convolutional layers for feature extraction and bidirectional LSTM layers to capture temporal dependencies in lip movements, supporting accurate transcription of silent commands. Over 15 epochs, the model showed a consistent decrease in both training loss, from an initial 94.254 to 56.7217, and validation loss, reducing from 72.1666 to 52.6722, reflecting improvements in transcription accuracy as predictions evolved from placeholder outputs to partial correctness. While this small-scale model demonstrates feasibility, it lays the groundwork for advancing non-intrusive, real-time alert systems through deep learning-based lipreading. The novelty of this system lies in its application of lipreading for emergency assistance, bridging a critical gap in silent distress detection technology.