Scholarlink

In this work, we present a system for continuous recognition of mouth patterns in Japanese Sign Language (JSL) for better visual communication. The continuous identification of five Japanese vowels is done on RGB image sequences using neural networks, including 3DConv, CNN-LSTM, TimeSformer, and ST-GCN. The neural networks are trained and evaluated on a custom dataset with the total number of images in the classes representing mouth vowels equal to 6,694. The dynamics of mouth articulations are analyzed using Grad-CAM-based heatmaps, which are generated from 3DConv features.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Continuous Recognition of Mouth Patterns in Japanese Sign Language for Visual Communication

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yuika Umeda,
  • Nurzhigit Ongalov,
  • Grzegorz Sroka,
  • Sako Shinji,
  • Bogdan Kwolek

摘要

In this work, we present a system for continuous recognition of mouth patterns in Japanese Sign Language (JSL) for better visual communication. The continuous identification of five Japanese vowels is done on RGB image sequences using neural networks, including 3DConv, CNN-LSTM, TimeSformer, and ST-GCN. The neural networks are trained and evaluated on a custom dataset with the total number of images in the classes representing mouth vowels equal to 6,694. The dynamics of mouth articulations are analyzed using Grad-CAM-based heatmaps, which are generated from 3DConv features.