Over the last two years, with the rise of LLMs, researchers and developers worldwide have demonstrated how this technology can bring to life applications that were once thought of as mere science fiction. Development has been so accelerated that the famous Turing Test now seems pointless. In particular, the concept of meta-programming has emerged where multiple LLMs act as agents to create 2D applications automatically, which has been demonstrated by projects such as AutoDev and MetaGPT. It only becomes logical to think about moving this concept into a 3D world to allow for the automatic generation of 3D applications. This paper outlines the basic building blocks of how such an approach could be achieved for a Virtual Reality application using two VR case study applications that use LLMs for the automatic generation of content. There is really only one name that fits such a 3D application in which a user can create anything they wish. It is colloquially known as a HoloDeck, inspired by Ivan Sutherland’s Ultimate Display concept. This paper sets aside the haptic feedback and rendering issues for the moment and focuses on the underlying software implementations, in which LLMs are not only the interface but also the underlying methodologies used within the system. As Ivan Sutherland once stated, “With appropriate programming, such a display could literally be the Wonderland into which Alice walked.” this paper proposes that LLMs using an agent-based approach acting as both the underlying programming and interface model could be that appropriate programming.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Is It Time to Start Taking the Concept of a Holodeck Seriously?

  • Quinn Berryman,
  • Mikey Dolan,
  • Valentin Boutouria,
  • Gueric Thomas,
  • Alex Johnston,
  • Fox Fleischmann,
  • Xuanhui Xu,
  • Abraham G. Campbell

摘要

Over the last two years, with the rise of LLMs, researchers and developers worldwide have demonstrated how this technology can bring to life applications that were once thought of as mere science fiction. Development has been so accelerated that the famous Turing Test now seems pointless. In particular, the concept of meta-programming has emerged where multiple LLMs act as agents to create 2D applications automatically, which has been demonstrated by projects such as AutoDev and MetaGPT. It only becomes logical to think about moving this concept into a 3D world to allow for the automatic generation of 3D applications. This paper outlines the basic building blocks of how such an approach could be achieved for a Virtual Reality application using two VR case study applications that use LLMs for the automatic generation of content. There is really only one name that fits such a 3D application in which a user can create anything they wish. It is colloquially known as a HoloDeck, inspired by Ivan Sutherland’s Ultimate Display concept. This paper sets aside the haptic feedback and rendering issues for the moment and focuses on the underlying software implementations, in which LLMs are not only the interface but also the underlying methodologies used within the system. As Ivan Sutherland once stated, “With appropriate programming, such a display could literally be the Wonderland into which Alice walked.” this paper proposes that LLMs using an agent-based approach acting as both the underlying programming and interface model could be that appropriate programming.