Reinforcement Learning (RL) is a class of methods used to support decision-making. Traditional RL focuses on maximizing the cumulative rewards derived from an agent’s interactions with an environment. However, it often suffers from data scarcity, as interactions are costly and acquired gradually. Bayesian Reinforcement Learning (BRL) mitigates this problem by incorporating prior knowledge about the environment and updating it with new data, but it requires an explicit likelihood function, which is often unavailable in real-world scenarios. We propose a full Bayesian Reinforcement Learning (fBRL) strategy to evaluate posterior distributions for both environment parameters and optimal policies. Our approach is based on Likelihood-free Iterated Batch Importance Sampling (LF-IBIS), a novel algorithm that combines Approximate Bayesian Computation with Iterated Batch Importance Sampling. This method overcomes the need for an explicit likelihood and enables online posterior updates. We test our approach on a Response-Adaptive Randomization problem in clinical trials..