ICE-Cubed: Inpainting of Cinematographic Elements for Intelligent Context Expansion
摘要
Immersive display systems like ICE Theaters enhance viewer experience by extending the field of view. However, maintaining immersion is challenging due to peripheral vision sensitivity. Elements near screen edges can disrupt the experience, and are difficult to detect as they lack semantic meaning and are highly context-dependent. We propose a two-step framework to address this issue. We benchmarked state-of-the-art segmentation models on our application. The best one, BiRefNet, is retrained on a custom dataset to improve detection. To complete this detection phase, a morphological dilation operation enhances mask coverage, followed by a filter to temporally stabilize detection. Second, we use E2FGVI for video inpainting, ensuring smooth content extension, then RealBasicVSR for super-resolution. To validate our approach, detection masks are assessed through expert evaluation and state-of-the-art mask quality metrics. Our results show superior performance over existing methods, both quantitatively and qualitatively. Finally, a perceptual study conducted with 5 ICE experts across 29 cinematic contents confirms our framework’s ability to enhance immersive cinema experiences.