We propose a different approach to generate deepfake videos based on Stable Diffusion, ControlNet, and Low-Rank Adaptation (LoRA). Stable Diffusion offers us greater control and fine-tuning options in the generation process. Compared to GANs, the proposed technique enables quick and easy modification of the obtained video by using a text prompt, adding or removing details, and altering the style and context of the deepfake. We describe the approach used and the generation pipeline, and then we show the application interface developed for the generation. Finally, we compare the quality of our deepfake generation framework with two other related approaches using two different tools that detect video/image manipulations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generating Deepfakes with Stable Diffusion, ControlNet, and LoRA

  • Stefano Bistarelli,
  • Francesco Santini,
  • Edoardo Toma Tavassi

摘要

We propose a different approach to generate deepfake videos based on Stable Diffusion, ControlNet, and Low-Rank Adaptation (LoRA). Stable Diffusion offers us greater control and fine-tuning options in the generation process. Compared to GANs, the proposed technique enables quick and easy modification of the obtained video by using a text prompt, adding or removing details, and altering the style and context of the deepfake. We describe the approach used and the generation pipeline, and then we show the application interface developed for the generation. Finally, we compare the quality of our deepfake generation framework with two other related approaches using two different tools that detect video/image manipulations.