TransLeakGAN: A Transformer-Based Framework for Enhanced Long Text Generation
摘要
This paper investigates advancements in Generative Adversarial Networks (GANs) for text generation. While GANs employing discriminative models as reinforcement learning policies have achieved promising results, they are limited by the use of a single scalar guidance signal available only after complete text generation. This drawback hinders the integration of valuable intermediate information regarding text structure throughout the generative process. LeakGAN introduced a solution by enabling the Discriminator to “leak” high-level features to the Generator during training. To improve upon LeakGAN’s performance, we propose TransLeakGAN, a novel algorithmic framework. TransLeakGAN incorporates Transformers as an alternative to the LSTM architecture within the Manager and Worker modules of the LeakGAN Generator. This substitution allows for a more powerful and nuanced approach to text generation. We conducted extensive experiments on both synthetic and real data, demonstrating that TransLeakGAN outperforms LeakGAN in the generation of long texts. Furthermore, this paper explores the influence of various loss functions and optimizers on the quality of generated sentences, aiming to achieve the most realistic text production possible through adversarial training.