This article presents a new method for leveraging explainable reinforcement learning (XRL) knowledge to enhance the performance of reinforcement learning (RL) agents. Although current XRL approaches are mainly focused on improving interpretability and user trust by providing explanations for agent actions, their ability to guide and optimise RL agent’s training is under-explored. To address this gap, we extend an existing introspective analysis framework by integrating XRL metrics directly into the training pipelines of model-free RL algorithms. This integration allows dynamic adjustments of algorithm-specific parameters based on real-time feedback from XRL metrics. The proposed methodology is validated across diverse OpenAI Gym environments (CartPole and Taxi). By evaluating both on-policy and off-policy approaches, we demonstrate that incorporating XRL insights leads to significant improvements in agent performance. The analysis of the results highlights the benefits regarding enhanced explainability and optimised decision-making. This work contributes in XRL research area by aligning interpretability with actionable performance gains, paving the way for more reliable and transparent RL systems in complex, real-world applications.