<p>In today’s digital era, multimedia content is increasing exceptionally. Due to this, there is a need for a solution that bridges the gap between linguistic barriers and globalization. The existing solution is unable to afford real-time translation and respective synchronization for sundry audiences. This study aims to develop a system named Vividh-Vaani, an innovative system that automates the process of translation and synchronization of video content in real-time. The system also pursues to boost multilingual approachability by exhorting the boundaries of existing methods and contributing to the user-centric experience. The system influences Automatic Speech Recognition (ASR) for extracting audio content and uses Neural Machine Translation (NMT) for high-quality language translation. The system also assimilated the Audio–Visual Temporal Alignment (AVTA) model for lip synchronization with ASR and NMT to ensure the unified alignment amid audio and visual constituents, augmenting the inspecting experience. The empirical evaluations demonstrates that the proposed framework achieves loftier performance over baseline systems such as Google Translate + TTS and Lecture Translator, with the lowest average WER (10.52%), TER (21.94%), and the highest BLEU (86.22%) and METEOR (0.78)s. From the obtained results, it concludes that the proposed Vividh-Vaani efficaciously conveys synchronized subtitles and dubbed audio in various languages, empowering users to access multimedia content in their preferred language. The proposed system’s cutting-edge features pointedly improve accessibility for educational contents, beating current market offerings. The proposed research validates Vividh-Vaani’s transformative ability to modernize multimedia ingesting in a digital era. By providing real-time, multilingual solutions, the research contributes to bridging language gaps, nurturing inclusivity, and boosting the ample extent of multimedia content in the field of education.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Vividh-Vaani: Video Translation and Synchronization using Machine Learning

  • Kavita Tukaram Patil,
  • Nita Jayesh Mahale,
  • Mayuri Diwakar Kulkarni,
  • Makarand Shahade,
  • Ashish Awate,
  • Bhushan Nandwalkar

摘要

In today’s digital era, multimedia content is increasing exceptionally. Due to this, there is a need for a solution that bridges the gap between linguistic barriers and globalization. The existing solution is unable to afford real-time translation and respective synchronization for sundry audiences. This study aims to develop a system named Vividh-Vaani, an innovative system that automates the process of translation and synchronization of video content in real-time. The system also pursues to boost multilingual approachability by exhorting the boundaries of existing methods and contributing to the user-centric experience. The system influences Automatic Speech Recognition (ASR) for extracting audio content and uses Neural Machine Translation (NMT) for high-quality language translation. The system also assimilated the Audio–Visual Temporal Alignment (AVTA) model for lip synchronization with ASR and NMT to ensure the unified alignment amid audio and visual constituents, augmenting the inspecting experience. The empirical evaluations demonstrates that the proposed framework achieves loftier performance over baseline systems such as Google Translate + TTS and Lecture Translator, with the lowest average WER (10.52%), TER (21.94%), and the highest BLEU (86.22%) and METEOR (0.78)s. From the obtained results, it concludes that the proposed Vividh-Vaani efficaciously conveys synchronized subtitles and dubbed audio in various languages, empowering users to access multimedia content in their preferred language. The proposed system’s cutting-edge features pointedly improve accessibility for educational contents, beating current market offerings. The proposed research validates Vividh-Vaani’s transformative ability to modernize multimedia ingesting in a digital era. By providing real-time, multilingual solutions, the research contributes to bridging language gaps, nurturing inclusivity, and boosting the ample extent of multimedia content in the field of education.