Multimodal Media Creation: Integrating LLMs for High-Quality Video Generation
摘要
The field of text-to-video generation is going through a transformation, by the advancements in Artificial Intelligence (AI) and deep learning. AI-powered video generation models help us in the conversion of textual descriptions into visual content, unlocking new possibilities in the fields of education, entertainment, and multimedia content creation. But the existing systems face challenges such as maintaining temporal coherence, such as ensuring smooth transitions, and correctly aligning video sequences with advanced textual inputs. This research combines Large Language Models (LLMs) and Generative Adversarial Networks (GANs) to develop a high-quality, temporally consistent text-to-video generation framework. The system uses the Gemini model to change textual prompts into detailed and structured descriptions. These descriptions are initially given to the Stable Diffusion model to get the corresponding text-image before being input into a fine-tuned MoCoGAN model for video synthesis. The VATEX dataset, which has extensive video and text pairs, is the primary training resource, ensuring meaningful alignment between textual descriptions and generated visuals. Apart from that, the research also explores the DAMO ViLab, which is a diffusion model that operates without additional training, providing a comparative analysis of different generative approaches. The results show enhanced video smoothness, improved scene consistency, and stronger semantic alignment with the textual prompts. This work advances text-to-video generation by addressing key limitations, using AI-driven storytelling and visualization applications.