Scholarlink

This study presents “Gaze Speak,” a novel multimodal human-computer interaction (HCI) system integrating AI-enabled eye-tracking and speech recognition for hands-free desktop control. Unlike traditional mouse/keyboard input, the proposed system leverages neural computer vision (Mediapipe), robust speech-to-text, and intelligent synchronization to master real-time control on commodity hardware. A rigorous experimental protocol with 20 participants, including users with disabilities, validates the system’s accuracy, flexibility, and accessibility impact. Extensive evaluation across lighting, noise, and multitasking environments reveals competitive latency, high recognition rates, and enthusiastic user feedback. Both quantitative and qualitative analyses, accompanied by data tables and architectural figures, demonstrate the promise of “Gaze Speak” for advancing usability and inclusive computing.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Gaze Speak: A Multimodal AI-Driven Human-Computer Interaction System Using Eye Gaze and Voice Commands

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • A. Aruna,
  • V. Aparna,
  • Anisha Jerin,
  • S. Subalakshmi

摘要

This study presents “Gaze Speak,” a novel multimodal human-computer interaction (HCI) system integrating AI-enabled eye-tracking and speech recognition for hands-free desktop control. Unlike traditional mouse/keyboard input, the proposed system leverages neural computer vision (Mediapipe), robust speech-to-text, and intelligent synchronization to master real-time control on commodity hardware. A rigorous experimental protocol with 20 participants, including users with disabilities, validates the system’s accuracy, flexibility, and accessibility impact. Extensive evaluation across lighting, noise, and multitasking environments reveals competitive latency, high recognition rates, and enthusiastic user feedback. Both quantitative and qualitative analyses, accompanied by data tables and architectural figures, demonstrate the promise of “Gaze Speak” for advancing usability and inclusive computing.