People intuitively teach robots by physically demonstrating actions, especially for complex in-contact manipulation tasks such as scrubbing, writing, or inserting objects. This study introduces a novel Act-it-Out method in which participants directly guide a 7-DOF robot arm while providing verbal instructions, treating the robot as a data collection instrument. Through this method, we collected a multimodal dataset of 360 physically demonstrated trajectories and 155 verbal commands across varied task types and clutter conditions. We analyzed the data using Laban Movement Analysis (LMA) Effort features—Force, Space, and Time—to explore how humans express motion intent through both physical and verbal channels. Findings reveal that while spatial and temporal cues often appear in speech, critical force information is primarily conveyed somatically, underscoring a gap in semantic-somatic alignment. We also examined how coaching prompts influenced instructional style, showing that priming affected participants’ emphasis on physical versus verbal guidance. This work contributes: (1) a novel Act-it-Out method for HRI data collection, (2) analysis of how motion qualities are distributed across somatic and semantic modes, (3) insights into coaching effects on communication, (4) a structured LMA-based feature space for learning from demonstration, and (5) a public dataset to support socially-aware robot learning from multimodal human input.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Act-it-Out Method for Developing Robot Arm Actions and Semantic Commands

  • Luke Sanchez,
  • Chirag Jain,
  • Shrirang Patil,
  • Bessie He,
  • Heather Knight

摘要

People intuitively teach robots by physically demonstrating actions, especially for complex in-contact manipulation tasks such as scrubbing, writing, or inserting objects. This study introduces a novel Act-it-Out method in which participants directly guide a 7-DOF robot arm while providing verbal instructions, treating the robot as a data collection instrument. Through this method, we collected a multimodal dataset of 360 physically demonstrated trajectories and 155 verbal commands across varied task types and clutter conditions. We analyzed the data using Laban Movement Analysis (LMA) Effort features—Force, Space, and Time—to explore how humans express motion intent through both physical and verbal channels. Findings reveal that while spatial and temporal cues often appear in speech, critical force information is primarily conveyed somatically, underscoring a gap in semantic-somatic alignment. We also examined how coaching prompts influenced instructional style, showing that priming affected participants’ emphasis on physical versus verbal guidance. This work contributes: (1) a novel Act-it-Out method for HRI data collection, (2) analysis of how motion qualities are distributed across somatic and semantic modes, (3) insights into coaching effects on communication, (4) a structured LMA-based feature space for learning from demonstration, and (5) a public dataset to support socially-aware robot learning from multimodal human input.