This article presents a review of modern approaches to creating parallel corpora for sign language machine translation, including data collection and annotation, synchronization of texts and video recordings, training models based on deep neural networks and transformers, and evaluating translation quality using standard metrics. As part of the study, a parallel corpus consisting of 998 sentences and over 5,000 words was developed, incorporating both manually generated and authentic texts from school textbooks. Particular attention is paid to analyzing technologies such as synthetic data generation, the application of multimodal features, and gesture visualization using 3D avatars. The primary goal of the study is to identify optimal strategies for creating corpora that account for the unique characteristics of sign languages and the limited availability of annotated data, thereby paving the way for the development of highly accurate machine translation systems that enhance information accessibility for deaf and hard-of-hearing individuals.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Current Trends in Developing Parallel Corpora for Text-to-Sign Language Translation

  • Nurzada Amangeldy,
  • Marek Milosz,
  • Aigerim Yerimbetova,
  • Nazira Tursynova,
  • Bekbolat Kurmetbek,
  • Nazerke Gazizova

摘要

This article presents a review of modern approaches to creating parallel corpora for sign language machine translation, including data collection and annotation, synchronization of texts and video recordings, training models based on deep neural networks and transformers, and evaluating translation quality using standard metrics. As part of the study, a parallel corpus consisting of 998 sentences and over 5,000 words was developed, incorporating both manually generated and authentic texts from school textbooks. Particular attention is paid to analyzing technologies such as synthetic data generation, the application of multimodal features, and gesture visualization using 3D avatars. The primary goal of the study is to identify optimal strategies for creating corpora that account for the unique characteristics of sign languages and the limited availability of annotated data, thereby paving the way for the development of highly accurate machine translation systems that enhance information accessibility for deaf and hard-of-hearing individuals.