Text-to-video generation with multi-loss sequential generative adversarial network
摘要
Generating videos from text remains challenging due to issues like structural complexity, semantic alignment, temporal coherence, and frame discontinuity. It is still highly difficult to generate realistic videos where the frames must comply to both spatial and temporal coherence, despite the fact that Generative Adversarial Networks (GANs) have been successfully deployed to generate videos conditioned on a text description. This study proposes a Sequential Generative Adversarial Network (Seq-GAN) that employs multiple generators and discriminators to generate realistic and high-quality videos using written descriptions, while maintaining semantic alignment and relevance between frames. The approach incorporates a modified Variational Autoencoder (VAE) with reduced and adjusted 2D layers to extract static “gist” features for improved performance. For dynamic frame generation, the model uses three generators (