<p>Sparse reward environments pose significant challenges for reinforcement learning, particularly in robotic navigation tasks, often leading to slow convergence and inefficient exploration. To address this, we propose a reward shaping method based on heat diffusion, governed by the heat equation, to propagate goal reward values throughout a discretized environment. The goal is modeled as a strong positive source, whereas obstacles act as weak low heat sinks, yielding a smooth reward field that preserves meaningful gradients around obstructions. A nonlinear power transformation and Gaussian smoothing are applied to refine the reward structure and enhance guidance near obstacles. This dense and continuous-reward landscape provides informative feedback at every step of training, mitigates the cold-start problem, and accelerates policy improvement. We integrated this shaping method into the Soft Actor-Critic (SAC) algorithm and evaluated it on two navigation environments with different obstacle geometries, using a differential-drive robot. The experiments covered four initialization scenarios that combined fixed and random robot start configurations and goal positions. The results show that heat diffusion-based reward shaping substantially improves learning speed and success rates compared to sparse rewards. In particular, variants incorporating reward filtering yield the most consistent gains by discouraging setbacks and encouraging directed motion. Although reward decay was also evaluated, its benefits were limited and task-dependent. In general, heat diffusion reward fields offer a principled and efficient approach to shaping rewards in sparse environments, enhancing reinforcement learning performance in robotic navigation tasks.</p>