This paper proposes a novel liquid cooling control strategy based on the Deep Diffusion Soft Actor-Critic (D2SAC) algorithm. Traditional temperature control methods often rely on manual modeling and lack adaptability, limiting their performance under dynamic operating conditions. To address this challenge, we construct a reinforcement learning training environment by integrating experimental data, Computational Fluid Dynamics (CFD) simulations, and simplified models, enabling real-time perception of system states such as battery temperature, voltage, and coolant flow. The D2SAC algorithm incorporates a diffusion model to generate fine-grained action sequences and employs a multilayer perceptron-based environment model to enhance policy learning, thereby improving both policy exploration and control precision. Experimental results demonstrate that the proposed method maintains battery temperatures within the desired range more stably, while reducing deviation and control costs. Compared with SAC, PPO, and threshold-based control, D2SAC achieves superior thermal regulation and energy efficiency, offering a promising solution for intelligent thermal management of energy storage systems.