A safe auxiliary feeding approach based on multimodal intention recognition and flexible interaction
摘要
For long-term bedridden individuals with limited mobility, comfortable auxiliary feeding is essential to improving quality of life. Although many feeding assistive robots have been developed, most still rely on fixed trajectories, forcing users to extend their necks, which is unfriendly to those with mobility limitations. In response, this study proposes a safe auxiliary feeding approach based on multimodal intention recognition and flexible interaction, aimed at improving the safety and convenience of diet for care recipients. The approach is vision-centered and combines haptic perception and interaction power to recognize multiple intentions of care recipients and provide adaptive meal assistance to efficiently complete food recognition, localization, grasping and delivery. Firstly, a novel framework for a meal assistance robotic system is established, and a multimodal interaction strategy is proposed. Then, to address the challenge of intent recognition during the meal process, algorithms are designed for meal intent recognition, meal selection intent recognition, and meal state recognition. The process requires only minimal actions" mouth opening and eye movement" for efficient meal assistance. In addition, a mouth pose recognition algorithm is proposed, enabling adaptive control of the robotic arm during the food delivery process. Finally, to enhance the safety and accuracy of the meal assistance process, trajectory planning is implemented. Through proof-of-concept dining experiments with multiple participants, the system demonstrated its potential to perform assisted feeding in a convenient and safe manner, particularly for individuals who are bedridden for long periods and have limited mobility.