Scholarlink

<p>Scene rearrangement is a crucial capability for household robotic assistants, requiring an embodied agent to restore objects to a previously recorded configuration after external modifications. This work addresses the core challenges of scene understanding and high-level task strategy , which depend on an agent’s ability to accurately identify objects, ascertain their states, and detect discrepancies from a target layout. Despite recent progress, a gap remains for methods that can operate without privileged information, such as complete scene geometry or ground truth object states. This paper presents a complete, end-to-end framework designed for such realistic constraints in complex kitchen environments. The proposed methodology integrates deep learning-based perception with spatio-temporal analysis of egocentric RGB-D data. This enables the system to detect object relocations, identify intrinsic state changes, infer necessary tool-based actions, and plan an optimal execution sequence for the entire task. Evaluated in the AI2-THOR simulation environment, the system achieves a 93.6% success rate on the FixedStrict metric and a 98.6% accuracy in inferring and executing state transformations, showcasing its robustness on complex, multi-step tasks under sequential observation constraints.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Embodied AI for kitchen scene rearrangement: a deep spatio-temporal approach

  • 期刊论文

  • 发表日期：

    2026年6月18日

查看全文
  • Arash Nasr Esfahani,
  • Hamed Hosseini,
  • Mehdi Tale Masouleh,
  • Ahmad Kalhor

摘要

Scene rearrangement is a crucial capability for household robotic assistants, requiring an embodied agent to restore objects to a previously recorded configuration after external modifications. This work addresses the core challenges of scene understanding and high-level task strategy , which depend on an agent’s ability to accurately identify objects, ascertain their states, and detect discrepancies from a target layout. Despite recent progress, a gap remains for methods that can operate without privileged information, such as complete scene geometry or ground truth object states. This paper presents a complete, end-to-end framework designed for such realistic constraints in complex kitchen environments. The proposed methodology integrates deep learning-based perception with spatio-temporal analysis of egocentric RGB-D data. This enables the system to detect object relocations, identify intrinsic state changes, infer necessary tool-based actions, and plan an optimal execution sequence for the entire task. Evaluated in the AI2-THOR simulation environment, the system achieves a 93.6% success rate on the FixedStrict metric and a 98.6% accuracy in inferring and executing state transformations, showcasing its robustness on complex, multi-step tasks under sequential observation constraints.