There is a scarcity of research in Visual Lip Reading (VLR) for non-English language, especially in Indian languages. Furthermore, VLR in low-resource Indian languages like Assamese has not been attempted earlier. This work is the first attempt to develop a VLR dataset for Assamese Compound Numeric Sequence Recognition. The primary contribution involves the construction of the VLRASN-112 dataset, comprising of 112 word categories for Assamese numbers, including both single and multiple digits. The dataset comprises of 300 unique numeric sequences recorded from 30 speakers in diverse environments. The second contribution is the proposal of a novel network architecture for the VLR task in the Assamese language, integrating Gabor features and attention mechanisms. The proposed model is compared against the widely used LipNet and LCSNet. Its performance is evaluated through Character Error Rate (CER) and Word Error Rate (WER) achieving CER of 0.39 and WER of 0.68.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VLRASN-112: A Visual Lip Reading Dataset for Assamese Compound Numeric Sequence Recognition

  • Meghali Deka,
  • Vaibhav Gavit,
  • Prithwijit Guha,
  • Sukumar Nandi,
  • Priyankoo Sarmah

摘要

There is a scarcity of research in Visual Lip Reading (VLR) for non-English language, especially in Indian languages. Furthermore, VLR in low-resource Indian languages like Assamese has not been attempted earlier. This work is the first attempt to develop a VLR dataset for Assamese Compound Numeric Sequence Recognition. The primary contribution involves the construction of the VLRASN-112 dataset, comprising of 112 word categories for Assamese numbers, including both single and multiple digits. The dataset comprises of 300 unique numeric sequences recorded from 30 speakers in diverse environments. The second contribution is the proposal of a novel network architecture for the VLR task in the Assamese language, integrating Gabor features and attention mechanisms. The proposed model is compared against the widely used LipNet and LCSNet. Its performance is evaluated through Character Error Rate (CER) and Word Error Rate (WER) achieving CER of 0.39 and WER of 0.68.