Nighttime deraining presents unique technical challenges due to complex degradation patterns, including low illumination and intertwined rain streaks that often amplify visual distortions. Existing methods typically formulate the rainy image as the linear combination of the rain streak residues and the clean image, disregarding the nonlinear interactions between rain and low-light conditions. Additionally, the mixed-scale representations of coupled degradations in nighttime rainy scenes have not been sufficiently explored to enhance deraining performance. To address these issues, we propose a mixed-scale Transformer that effectively captures multi-scale degradations and models them with enhanced nonlinearity, thereby formulating nighttime deraining as a nonlinear restoration problem. Specifically, we achieve mixed-scale representations of degradations through both intra-scale and inter-scale feature fusion across stages. We also propose a gated self-attention (GSA) mechanism to enhance the nonlinear modeling of the Transformer blocks. Building on the dense Mixture of Experts (MoE), we develop a dual-MoE feed-forward network (DMFN) that integrates a scale MoE (SMoE) and a nonlinear MoE (NMoE) to facilitate multi-scale and nonlinear feature representations, respectively. Extensive experimental results demonstrate that our method outperforms state-of-the-art methods on both synthetic and real-world datasets. The code is available at https://github.com/tandaily/NDformer .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

NDFormer: A Mixed-Scale Transformer with Enhanced Nonlinearity for Nighttime Image Deraining

  • Zhirui Liu,
  • Shangquan Sun,
  • Yuning Cui,
  • Dehong Kong,
  • Cong Zhang,
  • Wenqi Ren,
  • Kin-Man Lam

摘要

Nighttime deraining presents unique technical challenges due to complex degradation patterns, including low illumination and intertwined rain streaks that often amplify visual distortions. Existing methods typically formulate the rainy image as the linear combination of the rain streak residues and the clean image, disregarding the nonlinear interactions between rain and low-light conditions. Additionally, the mixed-scale representations of coupled degradations in nighttime rainy scenes have not been sufficiently explored to enhance deraining performance. To address these issues, we propose a mixed-scale Transformer that effectively captures multi-scale degradations and models them with enhanced nonlinearity, thereby formulating nighttime deraining as a nonlinear restoration problem. Specifically, we achieve mixed-scale representations of degradations through both intra-scale and inter-scale feature fusion across stages. We also propose a gated self-attention (GSA) mechanism to enhance the nonlinear modeling of the Transformer blocks. Building on the dense Mixture of Experts (MoE), we develop a dual-MoE feed-forward network (DMFN) that integrates a scale MoE (SMoE) and a nonlinear MoE (NMoE) to facilitate multi-scale and nonlinear feature representations, respectively. Extensive experimental results demonstrate that our method outperforms state-of-the-art methods on both synthetic and real-world datasets. The code is available at https://github.com/tandaily/NDformer .