Images captured with Under Display Camera (UDC) technology often experience various quality issues due to the inherent limitations of the capturing mechanism. For UDC image restoration, deep architectures utilizing CNNs and transformers frequently struggle to produce high-quality reconstructions because these networks often cannot effectively manage large receptive fields due to their inherent constraints. The recently proposed Mamba architecture, which employs State Space Models, has demonstrated promising results across various vision tasks, including image restoration applications such as denoising and super resolution. The model efficiently manages large receptive fields with linear time computational complexity. In this study, we evaluate the Mamba Model’s performance on UDC image restoration tasks after introducing a UDC specific additional module in the base architecture namely MambaIR. The proposed model, named UDC-Mamba, consists of a shallow restoration module, a novel hybrid deep enhancement module, and a selective scan module for high quality reconstruction. In our proposed hybrid deep enhancement module, convolutional blocks with multiple kernel sizes are used in conjunction with state space blocks. Experiments reveal that this model effectively restores UDC images, achieving notably superior perceptual quality compared to existing state-of-the-art methods. Our code is available at https://github.com/J-Karthik-palaniappan/UDC_Mamba

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

UDC-Mamba: Deep State Space Model for Under Display Camera Image Restoration

  • Aniruth Sundararajan,
  • MS Levin,
  • Karthik Palaniappan,
  • Jiji CV

摘要

Images captured with Under Display Camera (UDC) technology often experience various quality issues due to the inherent limitations of the capturing mechanism. For UDC image restoration, deep architectures utilizing CNNs and transformers frequently struggle to produce high-quality reconstructions because these networks often cannot effectively manage large receptive fields due to their inherent constraints. The recently proposed Mamba architecture, which employs State Space Models, has demonstrated promising results across various vision tasks, including image restoration applications such as denoising and super resolution. The model efficiently manages large receptive fields with linear time computational complexity. In this study, we evaluate the Mamba Model’s performance on UDC image restoration tasks after introducing a UDC specific additional module in the base architecture namely MambaIR. The proposed model, named UDC-Mamba, consists of a shallow restoration module, a novel hybrid deep enhancement module, and a selective scan module for high quality reconstruction. In our proposed hybrid deep enhancement module, convolutional blocks with multiple kernel sizes are used in conjunction with state space blocks. Experiments reveal that this model effectively restores UDC images, achieving notably superior perceptual quality compared to existing state-of-the-art methods. Our code is available at https://github.com/J-Karthik-palaniappan/UDC_Mamba