Recent trends in Reinforcement Learning have led to a proliferation of studies that applied it in diverse domains such as finance, in trading particularly. However, RL can be adversely affected under certain conditions. It is well established that financial markets are inherently noisy environments, where signals are often obscured by randomness, external shocks and non-stationary dynamics. However, few writers have been able to draw systematic research about how the presence of uncertainty and noise affect the performance of RL agents. The aim of this paper is to bridge the gap between the domains of RL and Economics. Specifically, we critically examine the impact that these challenges have on RL supported financial trading agents, exploring the affected performance, stability and robustness of these models. We discuss the key challenges financial noise and uncertainty impose, existing strategies and the limitations of currents models. This study aims to contribute to a deeper understanding of the challenges faced by RL in financial trading and proposes future directions for improving the agent’s performance in this context, including robust learning, model interpretability and domain knowledge integration.