This paper presents a method for combining Denoising Diffusion Probabilistic Models (DDPM) with Active Inference (AIF). The key idea is to leverage DDPM as a behavioural prior for inference within the AIF framework. This work is a novel application of DDPM to create a prior over actions, observations, and states for efficient planning and decision making. The DDPM is trained on behavourial observations in a training environment. Once trained, the DDPM can be conditioned on observations from a new environment without the need for retraining. The agent is then equipped to imagine likely future scenarios and focus on regions of the space which are most relevant. We show that estimates of the probability of future states can be used to provide inexpensive estimates of the Expected Free Energy. In addition, as the dimensions of the state space increase, the computational cost of generating policies is offset by a reduction in the number of policies to be considered. Together, these advantages overcome a major computational bottleneck. In summary, behavioural observations are efficiently encoded as data to train a DDPM, providing a simple but powerful prior for behaviour in a new environment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Diffusion Prior for Active Inference Planning

  • Catherine F. Higham,
  • Roderick Murray-Smith

摘要

This paper presents a method for combining Denoising Diffusion Probabilistic Models (DDPM) with Active Inference (AIF). The key idea is to leverage DDPM as a behavioural prior for inference within the AIF framework. This work is a novel application of DDPM to create a prior over actions, observations, and states for efficient planning and decision making. The DDPM is trained on behavourial observations in a training environment. Once trained, the DDPM can be conditioned on observations from a new environment without the need for retraining. The agent is then equipped to imagine likely future scenarios and focus on regions of the space which are most relevant. We show that estimates of the probability of future states can be used to provide inexpensive estimates of the Expected Free Energy. In addition, as the dimensions of the state space increase, the computational cost of generating policies is offset by a reduction in the number of policies to be considered. Together, these advantages overcome a major computational bottleneck. In summary, behavioural observations are efficiently encoded as data to train a DDPM, providing a simple but powerful prior for behaviour in a new environment.