Industrial robotic arm plays crucial role in manufacturing industry for effective production. However, the typical industrial robotic arms are not suitable for human–robot collaboration environment due to the lack of sensors and safety features to perceive the surrounding environment. This paper aims to design a motion control mechanism for industrial robotic arm by integrating a comprehensive facial expression recognition system. The vision system allows the industrial robotic arm to react with the corresponding changes in facial expression of humans for better human–robot interaction. A 3-layer system architecture was developed including the convolutional neural network facial expression recognition system that acted as the input, a Raspberry Pi as the microcomputer for decision-making process, and a custom-made 4-degree-of-freedom Arduino-based robotic arm as the motion output. A series of sequence emotions was used as input signals to evaluate the performance of the proposed mechanism in adjusting the robot motion speed with corresponding emotion. Simulation results demonstrated that the proposed mechanism is able to control the speed of motion of the robot corresponding to changes of facial expression and further validated through the considered Arduino-based robotic arm. This work demonstrated the potential of significant improvements in human–robot collaboration through the incorporation of the proposed motion control mechanism, leading to a more flexible, safe, and effective collaboration. For future works, a multimodal perception (FER) system could be integrated and more accurate deep learning models could be utilized to further improve the accuracy of facial expression recognition, leading to smoother responses from robots.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Motion Control of Arduino-Based Robotic Arm Using Facial Expression Recognition Approach for Human–Robot Collaboration

  • Chong Keat How,
  • Tsung Heng Chiew,
  • Yeh Huann Goh,
  • Kar Mun Chin,
  • Choan Xian How,
  • Jay Min Beh,
  • Yan Kai Tan,
  • Ting Sy Horng Terence

摘要

Industrial robotic arm plays crucial role in manufacturing industry for effective production. However, the typical industrial robotic arms are not suitable for human–robot collaboration environment due to the lack of sensors and safety features to perceive the surrounding environment. This paper aims to design a motion control mechanism for industrial robotic arm by integrating a comprehensive facial expression recognition system. The vision system allows the industrial robotic arm to react with the corresponding changes in facial expression of humans for better human–robot interaction. A 3-layer system architecture was developed including the convolutional neural network facial expression recognition system that acted as the input, a Raspberry Pi as the microcomputer for decision-making process, and a custom-made 4-degree-of-freedom Arduino-based robotic arm as the motion output. A series of sequence emotions was used as input signals to evaluate the performance of the proposed mechanism in adjusting the robot motion speed with corresponding emotion. Simulation results demonstrated that the proposed mechanism is able to control the speed of motion of the robot corresponding to changes of facial expression and further validated through the considered Arduino-based robotic arm. This work demonstrated the potential of significant improvements in human–robot collaboration through the incorporation of the proposed motion control mechanism, leading to a more flexible, safe, and effective collaboration. For future works, a multimodal perception (FER) system could be integrated and more accurate deep learning models could be utilized to further improve the accuracy of facial expression recognition, leading to smoother responses from robots.