We investigate the effect of dense residual connections in generative adversarial networks up to 20,000 epochs. Each consequent block layer receives all previous outputs, which are joined by an identity mapping before applying an activation function. In addition, we investigate the effect of using real label smoothing and label flipping of real and fake labels during the training of the discriminator. Using batch normalization layers after each convolution layer, in the discriminator or the generator, results in model instability leading to convergence failure or mode collapse which never manages to recover image generation. In the experimental runs that don’t use batch normalization layers, there is evidence of mode collapse and checkerboard artifacts early in the training which disappears later in the training. Label flipping in the discriminator results in a model that stabilizes and recovers image generation earlier compared to the other models (epoch 2750; baseline model with no residual connections—epoch 5000, model with residual connections—8000, with label smoothing—4500). Overall, generative adversarial networks with dense residual connections don’t show much improvement over the baseline model when it comes to stable training. The training of the dense residual models requires about 4 times as long to train.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dense Residual Connections in Generative Adversarial Networks

  • Mihail Mihaylov,
  • Jorge Ramón Fonseca Cacho

摘要

We investigate the effect of dense residual connections in generative adversarial networks up to 20,000 epochs. Each consequent block layer receives all previous outputs, which are joined by an identity mapping before applying an activation function. In addition, we investigate the effect of using real label smoothing and label flipping of real and fake labels during the training of the discriminator. Using batch normalization layers after each convolution layer, in the discriminator or the generator, results in model instability leading to convergence failure or mode collapse which never manages to recover image generation. In the experimental runs that don’t use batch normalization layers, there is evidence of mode collapse and checkerboard artifacts early in the training which disappears later in the training. Label flipping in the discriminator results in a model that stabilizes and recovers image generation earlier compared to the other models (epoch 2750; baseline model with no residual connections—epoch 5000, model with residual connections—8000, with label smoothing—4500). Overall, generative adversarial networks with dense residual connections don’t show much improvement over the baseline model when it comes to stable training. The training of the dense residual models requires about 4 times as long to train.