A Comparative Evaluation of AI Tools for Voice Cloning and Lip Syncing in Multimedia Content Creation
摘要
The research aims to explore progressive methods related to multimedia technology and gives insight into open-source artificial intelligence tools pertinent to video lip syncing and voice recording. Two of the main objectives of this paper are to discuss the latest techniques in multimedia technology as well as to share related information regarding open-source AI technology on video lip syncing and voice cloning. First, the investigation attempts to interact with the cutting-edge of multimedia technology by looking at innovative approaches related to voice cloning and lip syncing. This paper will also thus serve as a comprehensive guide for researchers and practitioners, by applying the latest artificial intelligence technologies, to gain expertise in areas such as voice cloning and video lip sync. Finally, this research contributes to the democratization of state-of-the-art multimedia technology by using open-source tools to make solutions for synchronizing voice and video readily accessible. This paper is aimed at equipping multimedia practitioners with vital information and resources, thus enabling them to understand and avail themselves of these open-source AI technologies to realize better speech and video synchronization in their creative works. In these experiments, the study evaluates Wav2lip and OpenSeq2Seq in fields like lip sync and speech synthesis on the LRS2-BBC dataset. The speech synthesis correctness, lip synchronization accuracy, transcription accuracy, and user ratings were all better for OpenSeq2Seq than Wav2lip. This established that for the tasks involved in speech synthesis and lip synchronization, OpenSeq2Seq is a much more accurate tool compared to Wav2lip, especially for the LRS2-BBC dataset.