Artificial intelligence (AI) applied in Virtual Reality (VR) has emerged as a dominant interaction media in multiple application areas, thanks to its advantage of providing users with immersive experiences. Its use in rehabilitative medicine and psychiatry has made significant progress. This paper aims to make an extended reality application that can be used to help patients that have balance disorders. The idea behind the project was to make an application in which users try to catch falling apples controlling the movement of the basket by moving their head left and right with the Hololens device on their head. The application was made in Unity environment, using inputs for gaze, gesture, voice, spatial sound, spatial mapping. It was designed to be used on the Hololens device for the patients to improve their motor functions and reflexes, assisting them in managing balance disorders. The game was created in a 2D version featuring assets such as baskets, apples, rocks, and a background. The basket asset represents the tool that patients use to catch apples that are constantly falling from the rock asset that is moving continuously left and right. At the start of the game, a user has three baskets; when the user misses to catch an apple, the basket disappears, and the user continues to play until there are no more baskets left. In the bottom corners of the user’s view, the current score representing the current number of apples the user has managed to catch is displayed, and there is also a high score field representing the highest number of apples the user has managed to catch. AI can enhance personalization in VR games by analyzing player behavior and preferences in real-time, tailoring immersive experiences to individual gaming styles and preferences. Future work can involve patients’ improvement results while playing the game, using AI to develop more games to improve the effectiveness of the treatment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AI-Enhanced Extended Reality in Medicine

  • Filip Filipović,
  • Tijana Geroski,
  • Aleksandra Vulović,
  • Nenad Filipović

摘要

Artificial intelligence (AI) applied in Virtual Reality (VR) has emerged as a dominant interaction media in multiple application areas, thanks to its advantage of providing users with immersive experiences. Its use in rehabilitative medicine and psychiatry has made significant progress. This paper aims to make an extended reality application that can be used to help patients that have balance disorders. The idea behind the project was to make an application in which users try to catch falling apples controlling the movement of the basket by moving their head left and right with the Hololens device on their head. The application was made in Unity environment, using inputs for gaze, gesture, voice, spatial sound, spatial mapping. It was designed to be used on the Hololens device for the patients to improve their motor functions and reflexes, assisting them in managing balance disorders. The game was created in a 2D version featuring assets such as baskets, apples, rocks, and a background. The basket asset represents the tool that patients use to catch apples that are constantly falling from the rock asset that is moving continuously left and right. At the start of the game, a user has three baskets; when the user misses to catch an apple, the basket disappears, and the user continues to play until there are no more baskets left. In the bottom corners of the user’s view, the current score representing the current number of apples the user has managed to catch is displayed, and there is also a high score field representing the highest number of apples the user has managed to catch. AI can enhance personalization in VR games by analyzing player behavior and preferences in real-time, tailoring immersive experiences to individual gaming styles and preferences. Future work can involve patients’ improvement results while playing the game, using AI to develop more games to improve the effectiveness of the treatment.