With the end goal of improving energy efficiency and fostering sustainable behavior, users are increasingly looking towards heat pumps as a highly energy efficient means of fulfilling thermal demand. However, the control of such systems is of utmost importance in order to enable safe and efficient exploitation of such systems. Whilst the existing control solutions, either the traditional ones, such as PID controllers, or more advanced, such as model predictive control, are lacking the ability to learn from experience, reinforcement learning approaches are able to learn from previous control actions and improve over time. Hence, this study proposes a reinforcement learning-based approach utilizing the Deep deterministic policy gradient algorithm for smart temperature control of buffer tanks in HVAC systems. By leveraging continuous action spaces, the Deep deterministic policy gradient algorithm enables precise and adaptive control of a heat pump system, improving energy efficiency and maintaining stable temperature levels. The results of the presented simulations demonstrate that the reinforcement learning-based control policy achieves a root mean square error of 4.7 °C in maintaining the buffer tank temperature, proving the potential for real-world applications.