A survey on deep reinforcement learning for personalized treatment featuring challenges and a theoretical framework
摘要
Personalized treatment is a paradigm shift in the medical field that provides individualized treatment strategies based on comprehensive patient data including habitat, genomic, and medical profile, etc. Integrating this into the conventional clinical process involves collaborative strategic planning with stakeholders from diverse sectors. The scope of this survey is limited to studies that apply reinforcement learning, deep reinforcement learning, and multi-objective deep reinforcement learning to sequential decision making in personalized treatment. The review addresses these main questions: Why are reinforcement learning and its variants suitable for personalized treatment?How are these algorithms applied in clinical settings of various domains? How are states, actions and rewards defined? What are the requirements, challenges at every stage of model development for deploying these models in safety-critical healthcare applications? Early studies primarily focused on reinforcement learning with single objective, while more works have increasingly adopted deep variants to manage high-dimensional state-action spaces, and a sparse literature exists with multi-objective approach to address conflicting treatment options. Most existing research is concentrated on using publicly available MIMIC- III /IV dataset in ICU settings, oncology related resources including, The Cancer Genome Atlas, Genomics of Drug Sensitivity in Cancer, The Catalogue of Somatic Mutations In Cancer. To the best of our knowledge, this is the first survey to emphasize: (i) the need for multi-objective optimization with sequential personalized treatment planning, (ii) the integration of non-clinical objectives such as patient preferences, hospital infrastructure, insurance coverage, etc., and stake-holders into decision framework, (iii) the theoretical and conceptual model of a multi-modal multi-objective deep reinforcement learning aiming at optimizing the treatment regimes using a newly introduced time-weighted reward integration block to balance multiple, potentially conflicting clinical as well as non-clinical objectives.