Scholarlink

<p>Enabling intuitive, low-level robot control through natural spoken language is essential for deploying service robots in everyday environments. However, verbal commands are inherently variable and often ambiguous, making it difficult for robots to consistently interpret and execute them without extensive handcrafted rules or predefined templates. In this paper, we present the <i>directive language model</i> (DLM), a novel speech-to-trajectory framework that directly maps unconstrained verbal instructions to low-level robot motion trajectories. DLM is trained using behavior cloning (BC) on demonstrations in simulation, where human participants issue spoken guidance and control the robot’s motion accordingly. To improve generalization across phrasings, we apply semantic augmentation via GPT-4, generating diverse paraphrases that share the same trajectory labels. A text-conditioned diffusion policy is then employed to generate smooth and flexible trajectories that align with user intent. Unlike large language model (LLM)-based approaches that require prompt engineering and often yield unpredictable responses, DLM ensures consistent motion generation with efficient inference suitable for onboard deployment. Our results, validated in both simulation and on a quadruped robot, show that DLM robustly interprets a wide range of user commands, including paraphrased, truncated, and implicit instructions, without relying on perception or symbolic planning. These capabilities make DLM well-suited for natural, speech-based control of service robots interacting with untrained users.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Speech-to-trajectory: learning intuitive verbal guidance for robot motion

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月19日

查看全文
  • Eran Beeri Bamani,
  • Eden Nissinman,
  • Rotem Atari,
  • Nevo Heimann Saadon,
  • Avishai Sintov

摘要

Enabling intuitive, low-level robot control through natural spoken language is essential for deploying service robots in everyday environments. However, verbal commands are inherently variable and often ambiguous, making it difficult for robots to consistently interpret and execute them without extensive handcrafted rules or predefined templates. In this paper, we present the directive language model (DLM), a novel speech-to-trajectory framework that directly maps unconstrained verbal instructions to low-level robot motion trajectories. DLM is trained using behavior cloning (BC) on demonstrations in simulation, where human participants issue spoken guidance and control the robot’s motion accordingly. To improve generalization across phrasings, we apply semantic augmentation via GPT-4, generating diverse paraphrases that share the same trajectory labels. A text-conditioned diffusion policy is then employed to generate smooth and flexible trajectories that align with user intent. Unlike large language model (LLM)-based approaches that require prompt engineering and often yield unpredictable responses, DLM ensures consistent motion generation with efficient inference suitable for onboard deployment. Our results, validated in both simulation and on a quadruped robot, show that DLM robustly interprets a wide range of user commands, including paraphrased, truncated, and implicit instructions, without relying on perception or symbolic planning. These capabilities make DLM well-suited for natural, speech-based control of service robots interacting with untrained users.