Physics-guided deep reinforcement learning for personalized PDE control in cancer therapy optimization
摘要
A significant challenge in computational engineering is the optimal control of complex, nonlinear dynamical systems governed by partial differential equations (PDEs). This study presents a physics-guided deep reinforcement learning (DRL) framework designed to solve such a high-dimensional control problem in the context of personalized cancer therapy. Our approach integrates physics priors, a mechanistic PDE model capturing tumor-immune-drug interactions, with a model-free DRL agent tasked with simultaneously optimizing drug selection, dosage, and treatment duration. A key innovation is the agent’s ability to learn effective control policies without any prior knowledge of the underlying PDE system’s structure or parameters. The agent learns exclusively from state observations, representing spatial distributions of cell populations, and a reward function designed to minimize both tumor burden and treatment toxicity. We conducted a comprehensive evaluation comparing three prominent DRL algorithms: Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Trust Region Policy Optimization (TRPO). The agents were trained under conditions of simulated clinical heterogeneity to ensure the robustness and generalizability of the learned policies. Our results demonstrate that PPO, when configured with an adaptive entropy scheduling, achieves superior performance by effectively balancing aggressive tumor eradication with minimal toxicity. This analysis also underscores the critical role of the exploration-exploitation trade-off, showing how tuning the policy entropy or trust-region radius directly impacts convergence, policy stability, and the clinical viability of the resulting treatment strategy. This work highlights the potential of synergizing DRL with physics-based models to create powerful, adaptive decision-support systems for complex control problems in precision medicine and beyond.