This paper presents a voice-controlled Unmanned Aerial Vehicle (UAV) system that integrates Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) techniques to enhance command interpretation and execution. The proposed system was tested in a Gazebo and ROS-based simulation environment using the Iris drone model from ArduPilot SITL. Various ASR models, including Whisper (Tiny, Base, Large), Wav2Vec2 (Base, Large), and Google Speech Recognition, were evaluated for their ability to transcribe drone-related voice commands accurately. Additionally, Natural Language Inference (NLI) models such as Facebook/BART (Base and Large MNLI), Google/T5-Large, Electra-Large-Discriminator, and GPT-4o-mini were assessed for mapping transcribed commands to MAVLink messages. Experimental results demonstrate that Whisper Base outperforms other ASR models in balancing accuracy and processing efficiency, making it the best candidate for UAV speech recognition. Among the NLP/NLI models, BART-Large-MNLI exhibited the highest accuracy in correctly mapping voice commands to UAV actions, outperforming other models in zero-shot classification. The study highlights the feasibility of deploying local ASR models to reduce latency while maintaining high transcription accuracy and emphasizes the need for context-aware NLP models to enhance drone command flexibility. Future work will focus on expanding the dataset, optimizing real-time processing, and improving conversational UAV command execution.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Voice-Controlled Drone Navigation: A Hybrid Approach Using ASR and NLP for UAV Command Interpretation

  • Yassir Alkasim,
  • Abdulrahman Altahhan

摘要

This paper presents a voice-controlled Unmanned Aerial Vehicle (UAV) system that integrates Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) techniques to enhance command interpretation and execution. The proposed system was tested in a Gazebo and ROS-based simulation environment using the Iris drone model from ArduPilot SITL. Various ASR models, including Whisper (Tiny, Base, Large), Wav2Vec2 (Base, Large), and Google Speech Recognition, were evaluated for their ability to transcribe drone-related voice commands accurately. Additionally, Natural Language Inference (NLI) models such as Facebook/BART (Base and Large MNLI), Google/T5-Large, Electra-Large-Discriminator, and GPT-4o-mini were assessed for mapping transcribed commands to MAVLink messages. Experimental results demonstrate that Whisper Base outperforms other ASR models in balancing accuracy and processing efficiency, making it the best candidate for UAV speech recognition. Among the NLP/NLI models, BART-Large-MNLI exhibited the highest accuracy in correctly mapping voice commands to UAV actions, outperforming other models in zero-shot classification. The study highlights the feasibility of deploying local ASR models to reduce latency while maintaining high transcription accuracy and emphasizes the need for context-aware NLP models to enhance drone command flexibility. Future work will focus on expanding the dataset, optimizing real-time processing, and improving conversational UAV command execution.