Self-determined mobility is a central aspect of social participation. For people with motor impairments, such as those seen in tetraplegia, muscular dystrophy, cerebral palsy, multiple sclerosis, or ALS, operating conventional electric wheelchairs often poses a significant challenge. This work presents a system under development for controlling electric wheelchairs using eye and voice commands. The system is based on a Magic Leap 2 augmented reality headset, which uses eye tracking and speech recognition to capture the user’s intent. A virtual joystick, whose appearance is customizable, translates eye movements into control signals. Voice control enables additional functions such as turning the joystick on and off. The system is compatible with both a specially developed wheelchair simulator and a real electric wheelchair (Ottobock Juvo B5) via a Raspberry Pi as an interface. The communication is bidirectional via a USB connection. Future work includes the complete implementation of the connection to the real wheelchair, the improvement of the accessibility of the user interface, and especially comprehensive evaluation studies in collaboration with Ottobock and the BG Clinic Hamburg. The aim of the studies is to investigate the suitability of the combined eye and voice control as a complement to existing special controls and, if necessary, to identify limiting factors. The research is conducted in strict adherence to ethical guidelines.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Eye-Controlled Joystick with Voice Control for Operating a Simulated and Real Electric Wheelchair

  • Jendrik Bulk

摘要

Self-determined mobility is a central aspect of social participation. For people with motor impairments, such as those seen in tetraplegia, muscular dystrophy, cerebral palsy, multiple sclerosis, or ALS, operating conventional electric wheelchairs often poses a significant challenge. This work presents a system under development for controlling electric wheelchairs using eye and voice commands. The system is based on a Magic Leap 2 augmented reality headset, which uses eye tracking and speech recognition to capture the user’s intent. A virtual joystick, whose appearance is customizable, translates eye movements into control signals. Voice control enables additional functions such as turning the joystick on and off. The system is compatible with both a specially developed wheelchair simulator and a real electric wheelchair (Ottobock Juvo B5) via a Raspberry Pi as an interface. The communication is bidirectional via a USB connection. Future work includes the complete implementation of the connection to the real wheelchair, the improvement of the accessibility of the user interface, and especially comprehensive evaluation studies in collaboration with Ottobock and the BG Clinic Hamburg. The aim of the studies is to investigate the suitability of the combined eye and voice control as a complement to existing special controls and, if necessary, to identify limiting factors. The research is conducted in strict adherence to ethical guidelines.