<p>This paper addresses reflection removal, which is the task of separating reflection components from a captured image and deriving the image with only transmission components. Considering that the existence of the reflection changes the polarization state of a scene, some existing methods have exploited polarized images for reflection removal. While these methods apply polarized images as the inputs, they predict the reflection and the transmission directly as non-polarized intensity images. In contrast, we propose a polarization-to-polarization approach that applies polarized images as the inputs and predicts “polarized”reflection and transmission images using two sequential networks to facilitate the separation task by utilizing the interrelated polarization information between the reflection and the transmission. We further adopt a recurrent framework, where the predicted reflection and transmission images are used to iteratively refine each other. To address the lack of a color-polarized image dataset for reflection removal training, we propose a physics-based synthetic dataset generation pipeline designed to produce color-polarized images with reflections. Additionally, to evaluate the generalization capability for real-world scenes, we introduce a new real-world color test dataset captured using a polarization camera. Experimental results on existing grayscale and our color datasets demonstrate that our method outperforms other state-of-the-art approaches.</p>