Immersive display systems like ICE Theaters enhance viewer experience by extending the field of view. However, maintaining immersion is challenging due to peripheral vision sensitivity. Elements near screen edges can disrupt the experience, and are difficult to detect as they lack semantic meaning and are highly context-dependent. We propose a two-step framework to address this issue. We benchmarked state-of-the-art segmentation models on our application. The best one, BiRefNet, is retrained on a custom dataset to improve detection. To complete this detection phase, a morphological dilation operation enhances mask coverage, followed by a filter to temporally stabilize detection. Second, we use E2FGVI for video inpainting, ensuring smooth content extension, then RealBasicVSR for super-resolution. To validate our approach, detection masks are assessed through expert evaluation and state-of-the-art mask quality metrics. Our results show superior performance over existing methods, both quantitatively and qualitatively. Finally, a perceptual study conducted with 5 ICE experts across 29 cinematic contents confirms our framework’s ability to enhance immersive cinema experiences.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ICE-Cubed: Inpainting of Cinematographic Elements for Intelligent Context Expansion

  • David Traparic,
  • Maugan De Murcia,
  • Mohamed-Chaker Larabi,
  • Ladjel Bellatreche

摘要

Immersive display systems like ICE Theaters enhance viewer experience by extending the field of view. However, maintaining immersion is challenging due to peripheral vision sensitivity. Elements near screen edges can disrupt the experience, and are difficult to detect as they lack semantic meaning and are highly context-dependent. We propose a two-step framework to address this issue. We benchmarked state-of-the-art segmentation models on our application. The best one, BiRefNet, is retrained on a custom dataset to improve detection. To complete this detection phase, a morphological dilation operation enhances mask coverage, followed by a filter to temporally stabilize detection. Second, we use E2FGVI for video inpainting, ensuring smooth content extension, then RealBasicVSR for super-resolution. To validate our approach, detection masks are assessed through expert evaluation and state-of-the-art mask quality metrics. Our results show superior performance over existing methods, both quantitatively and qualitatively. Finally, a perceptual study conducted with 5 ICE experts across 29 cinematic contents confirms our framework’s ability to enhance immersive cinema experiences.