Deep reinforcement learning enabled intelligent decision-making methods for human-machine symbiosis manufacturing
摘要
Human-machine symbiosis manufacturing (HMSM) is an important enabler of smart manufacturing. However, complex constraints and disturbance events make it difficult to make reasonable decisions in such systems. Meanwhile, deep reinforcement learning (DRL) has emerged as a promising decision-making approach. Therefore, this paper develops DRL-enabled intelligent decision-making methods for HMSM, which mainly include a human-machine collaborative scheduling method and four auxiliary methods. In the scheduling method, a stacked multi-graph attention neural network is first designed using node-based graph decomposition and two neighborhood aggregation techniques. It can achieve accurate state extraction of the factory environment. Then, a dynamic scheduling method based on independent discrete proximal policy optimization is proposed, which incorporates an agent collaborative learning mechanism with reward sharing and independent updates. It enables end-to-end operation assignment and device combination. In addition, four auxiliary methods are designed to facilitate further decision-making. In particular, a factory adaptive growth method is developed based on bottleneck resource analysis and a device purchase algorithm. Furthermore, an intelligent decision-making software is implemented in a real-world precision equipment factory. Experimental results demonstrate that, compared with rules, genetic programming, and three DRL-based methods, the proposed decision-making methods exhibit superiority and generalization, and can ensure efficient production in dynamic environments.