This paper investigates advancements in Generative Adversarial Networks (GANs) for text generation. While GANs employing discriminative models as reinforcement learning policies have achieved promising results, they are limited by the use of a single scalar guidance signal available only after complete text generation. This drawback hinders the integration of valuable intermediate information regarding text structure throughout the generative process. LeakGAN introduced a solution by enabling the Discriminator to “leak” high-level features to the Generator during training. To improve upon LeakGAN’s performance, we propose TransLeakGAN, a novel algorithmic framework. TransLeakGAN incorporates Transformers as an alternative to the LSTM architecture within the Manager and Worker modules of the LeakGAN Generator. This substitution allows for a more powerful and nuanced approach to text generation. We conducted extensive experiments on both synthetic and real data, demonstrating that TransLeakGAN outperforms LeakGAN in the generation of long texts. Furthermore, this paper explores the influence of various loss functions and optimizers on the quality of generated sentences, aiming to achieve the most realistic text production possible through adversarial training.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TransLeakGAN: A Transformer-Based Framework for Enhanced Long Text Generation

  • Harsha Vasireddy,
  • Bhukya Krishna Priya,
  • B. Sivaselvan

摘要

This paper investigates advancements in Generative Adversarial Networks (GANs) for text generation. While GANs employing discriminative models as reinforcement learning policies have achieved promising results, they are limited by the use of a single scalar guidance signal available only after complete text generation. This drawback hinders the integration of valuable intermediate information regarding text structure throughout the generative process. LeakGAN introduced a solution by enabling the Discriminator to “leak” high-level features to the Generator during training. To improve upon LeakGAN’s performance, we propose TransLeakGAN, a novel algorithmic framework. TransLeakGAN incorporates Transformers as an alternative to the LSTM architecture within the Manager and Worker modules of the LeakGAN Generator. This substitution allows for a more powerful and nuanced approach to text generation. We conducted extensive experiments on both synthetic and real data, demonstrating that TransLeakGAN outperforms LeakGAN in the generation of long texts. Furthermore, this paper explores the influence of various loss functions and optimizers on the quality of generated sentences, aiming to achieve the most realistic text production possible through adversarial training.