Shadow-Based Touch Detection for Virtual Forearm Keyboards in AR Text Entry
摘要
This paper describes a text entry method with a virtual forearm keyboard in augmented reality (AR). A virtual QWERTY keyboard is overlaid on the user’s forearm, and text is entered by touching the skin with the opposite hand’s fingertip in an AR text entry setting. Fingertip-skin contact is detected using a camera mounted on a head-mounted display (HMD) without requiring additional devices, and a deep learning model analyzes fingertip shadow variations. This vision-based technique preserves user mobility and integrates seamlessly with other AR interaction methods. We implemented the method in a prototype system and evaluated it through a user study. Participants entered text, including high-entropy inputs such as passwords. Performance was measured using Raw Words Per Minute (Raw WPM), Character Error Rate (CER), while subjective satisfaction was assessed via the User Experience Questionnaire (UEQ). Despite minimal training, participants achieved an average Raw WPM of 3.96 words/minute, a CER of 0.23%, and a UEQ score of 1.17, indicating high usability and user satisfaction.