In the last decade, generative models have seen a significant amount of attention due to their remarkable capacity of generating almost any kind of multimedia content. However, these models are highly nonlinear, and their complexity does not allow an easy understanding of the reasoning behind their generative procedure, which is therefore characterized by a lack of explainability. Moreover, in some sensitive applicative fields like medical diagnosis or autonomous driving, even a little error can be dangerous.Recently, some methods to explain conventional deep learning models have been introduced and then extended to common generative models such as generative adversarial networks (GANs). However, few attempts have been proposed to explain and understand diffusion models, which are currently state-of-the-art methods for multimedia content generation. In this paper, we propose to take a step towards better explaining the generated output of a diffusion model by specifically conditioning how such a model makes its decision on the generation process. We propose to involve counterfactual visual explanations, a type of explainable information that is produced by editing a starting sample, to condition pretrained diffusion models. The result of the model will then be an edited image containing specific explainable attributes, which allows us to understand the decision of a neural classifier applied to such generated images. In the experimental evaluation on different datasets, we show how our diffusion explainable (DiffusionEx) model generates plausible and visually pleasant samples containing counterfactual attributes, while crucially increasing and explaining the correct classification probabilities of such images, according to a conventional classifier.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Counterfactual-Aware Diffusion Models

  • Federico Ambrogio,
  • Francesco Fè,
  • Luigi Sigillo,
  • Eleonora Grassucci,
  • Danilo Comminiello

摘要

In the last decade, generative models have seen a significant amount of attention due to their remarkable capacity of generating almost any kind of multimedia content. However, these models are highly nonlinear, and their complexity does not allow an easy understanding of the reasoning behind their generative procedure, which is therefore characterized by a lack of explainability. Moreover, in some sensitive applicative fields like medical diagnosis or autonomous driving, even a little error can be dangerous.Recently, some methods to explain conventional deep learning models have been introduced and then extended to common generative models such as generative adversarial networks (GANs). However, few attempts have been proposed to explain and understand diffusion models, which are currently state-of-the-art methods for multimedia content generation. In this paper, we propose to take a step towards better explaining the generated output of a diffusion model by specifically conditioning how such a model makes its decision on the generation process. We propose to involve counterfactual visual explanations, a type of explainable information that is produced by editing a starting sample, to condition pretrained diffusion models. The result of the model will then be an edited image containing specific explainable attributes, which allows us to understand the decision of a neural classifier applied to such generated images. In the experimental evaluation on different datasets, we show how our diffusion explainable (DiffusionEx) model generates plausible and visually pleasant samples containing counterfactual attributes, while crucially increasing and explaining the correct classification probabilities of such images, according to a conventional classifier.