Bio-inspired adaptive synergy control policy incorporated with reinforcement learning for posture control of a musculoskeletal model
摘要
The human musculoskeletal model has structural characteristics of high redundancy, strong coupling, and significant nonlinearity, making it extremely challenging to achieve efficient, accurate, and robust posture control. The traditional actor-critic reinforcement learning (ACRL) algorithm often faces problems such as low exploration efficiency, insufficient control accuracy, and poor disturbance rejection when dealing with high-dimensional muscle drive spaces. Inspired by the neural mechanism by which the biological corticospinal tract regulates muscle activity, this paper proposes an adaptive learning ensemble control method that integrates a muscle synergy control policy. In this method, the ACRL algorithm is combined with the muscle length feedback mechanism, in which the muscle group control policy and individual control policy are dynamically integrated, thereby optimizing the exploration efficiency of the high-dimensional action space, and accurate control of human upper limb posture is achieved. In the simulation experiment of a human right upper limb model containing 20 muscles and 5 degrees of freedom, the mean angular error for pose arrival tasks is reduced by approximately 20% under gravitational conditions, and the maximum deviation angle is reduced by approximately 40% under transient external disturbances, demonstrating good control accuracy and robustness. The proposed musculoskeletal model control method in this study validates the key role of muscle coordination in solving the control redundancy problem of musculoskeletal model and enhancing disturbance robustness, providing new insights for biomimetic motion control.