AI-Assisted Natural Programming of Assistive Robots Using Verbal Commanding with Assistive Robots for the Elderly – An Explorative Study Using IsaacSim and ChatGPT
摘要
The aging population is a global challenge, with the number of older adults rapidly increasing due to advancements in medicine and healthcare. However, the inevitable decline in physical abilities presents significant challenges for independent living and has led to a substantial rise in the demand for assisted living. It is widely anticipated that assistive robots will play a crucial role in providing effective solutions in the near future. However, there has been limited research on developing assistive robotic solutions that offer intuitive and user-friendly interfaces. This study seeks to explore the possibility of using large language models (LLMs) to operate and control mobile robots through spoken commands, providing a natural conversational approach tailored to the needs of older adults. The study introduces a robotic simulation environment utilizing NVIDIA IsaacSim and the Omniverse engine, where the effectiveness of robot control was evaluated based on the accuracy and responsiveness of the robots to verbal commands. The NVIDIA Jetbot was used to navigate a simulated environment filled with various objects and human models. In order to operate the Jebot, the necessary functions for the vehicle were labeled and provided to GPT as a prompt, enabling it to generate movement code for the robots based on the user’s spoken requests. GPT was able to understand verbal commands, currently typed into the terminal communicating with GPT, and run the appropriate movement code. However, regardless of the parameters set for GPT before operation, GPT was sometimes faulty in scripting its code or understanding the prompt. The findings of this experiment revealed the strong ability of LLMs to operate basic robots through spoken commands, which indicates the potential that LLMs could have in operating assistive robots for older adults.