This research explores the development of an advanced AI-driven dialogue simulator specifically designed to assist educators in effectively practicing and navigating various pedagogical scenarios through role-playing. The system integrates state-of-the-art generative AI with virtual reality (VR), utilizing dynamic 3D avatars capable of interactive dialogue and real-time question generation based on predefined knowledge and user input. Educators can engage in realistic simulations, receiving personalized and contextually relevant questions and feedback from AI-generated virtual interlocutors. By leveraging speech-to-text technology, user interactions are seamlessly processed to generate immediate, targeted responses, facilitating an immersive and responsive learning environment. This dialogue simulator aims to enhance educators’ preparedness, adaptability, and communication skills by allowing iterative practice in a safe virtual setting. Technical implementations include the utilization of open-source large language models, real-time avatar lip synchronization, customizable virtual personas, and sophisticated scenario generation methods. Initial evaluations were conducted to assess the feasibility and usability of the VR-based dialogue simulator. Specifically, exploring whether educators found the system immersive, realistic, and helpful for practicing pedagogical scenarios. Future research will further refine AI responsiveness and expand scenario complexity, contributing meaningfully to AI-assisted pedagogical training methodologies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Education Training with A Virtual Reality-Based Simulator Integrated Generative AI

  • Meisam Taheri,
  • Kevin Tan,
  • Ole Einar Flaten,
  • Magdalena Cyma-Wejchenig

摘要

This research explores the development of an advanced AI-driven dialogue simulator specifically designed to assist educators in effectively practicing and navigating various pedagogical scenarios through role-playing. The system integrates state-of-the-art generative AI with virtual reality (VR), utilizing dynamic 3D avatars capable of interactive dialogue and real-time question generation based on predefined knowledge and user input. Educators can engage in realistic simulations, receiving personalized and contextually relevant questions and feedback from AI-generated virtual interlocutors. By leveraging speech-to-text technology, user interactions are seamlessly processed to generate immediate, targeted responses, facilitating an immersive and responsive learning environment. This dialogue simulator aims to enhance educators’ preparedness, adaptability, and communication skills by allowing iterative practice in a safe virtual setting. Technical implementations include the utilization of open-source large language models, real-time avatar lip synchronization, customizable virtual personas, and sophisticated scenario generation methods. Initial evaluations were conducted to assess the feasibility and usability of the VR-based dialogue simulator. Specifically, exploring whether educators found the system immersive, realistic, and helpful for practicing pedagogical scenarios. Future research will further refine AI responsiveness and expand scenario complexity, contributing meaningfully to AI-assisted pedagogical training methodologies.