Camera and Metaverse: Enhancing User Experience Using AI-Based Imaging
摘要
The advent of the internet since the 1990s has shrunk the world to a global village. The progress of Moore’s law has allowed technological advancement from telephones to voice over internet protocol (VOIP) to metaverse thereby facilitating communication on an unprecedented scale. Metaverse primarily comprises of augmented reality (AR) and virtual reality (VR) devices that are used for interacting between the digital and physical worlds. Recent advancements in AR/VR imaging pipelines from ultra compact direct time-of-flight modules to folded optics and AI-based image processing is reshaping the future. Camera happens to be a pivotal centerpiece of AR/VR devices and is no longer considered just a sensor. The camera is now a portal that facilitates immersive and intelligent communication. Camera imaging technology has transformed from a tedious multi step analog chemical process to an almost instantaneous digital process with an abundance of image sharing possibilities. In VR headsets, “pass-through” permits users to view their real-world surroundings with cameras on the headset displaying stereoscopic, color, high resolution, low latency real time videos in the device. The cameras in AR glasses not only capture images and videos to share them in Facebook or Instagram but also simultaneously analyze visual data and communicate using artificial intelligence (AI), live translation, hand and motion tracking. AI with image processing, machine learning algorithms and deep learning architecture has significantly amplified the immersive experience in AR/VR devices and has enabled real life experience in the virtual domain. In this chapter we discuss the evolution of cameras, camera architecture, image processing techniques in head-mounted VR displays (HMDs)/AR glasses, machine learning algorithms and AI-based imaging techniques that have enabled fully immersive realistic user experiences.