<p>User-side resources (USRs), including flexible loads and distributed storage, are increasingly integrated into grid regulation. However, their inherent heterogeneity, response uncertainty, and geographically dispersed deployment pose significant challenges to unified and scalable control. Existing regulation strategies often struggle to simultaneously capture multi-layer coordination requirements and the dynamic flexibility boundaries of heterogeneous resources. To address this issue, this paper proposes a multi-layer collaborative control framework in which upper-layer decision-making is explicitly coupled with lower-layer flexibility intervals. Specifically, the upper-layer controller allocates regulation tasks across aggregated USRs, while the lower layer decomposes these tasks into resource-level actions constrained by dynamically evolving flexible intervals. The overall decision process is formulated as a Markov decision process (MDP), and a Deep Deterministic Policy Gradient (DDPG) algorithm is employed to learn optimal control policies for continuous decision spaces inherent in USR regulation. To ensure effective learning, a composite reward function is designed to jointly capture operating cost, regulation deviation, and constraint violation penalties, enabling the agent to balance economic efficiency and control accuracy. Extensive simulation studies, including convergence analysis, baseline comparisons, computational efficiency evaluation, ablation studies, and hyperparameter sensitivity analysis, are conducted to validate the proposed approach. The results demonstrate that the proposed framework significantly improves peak-shaving performance and operational economy, while achieving stable convergence and strong adaptability in complex multi-layer regulation scenarios.</p>