<p>Polarimetric imaging captures the vector properties of light, revealing rich physical information beyond conventional intensity-based methods. Its integration with deep learning has advanced visual perception in complex environments, yet the field remains fragmented, lacking a unified framework. Based on the polarimetric forward imaging process, this survey introduces a three-layer taxonomy (Signal, Physical, and Semantic) to systematically organize research advances. At the Signal Layer, we review methods for image restoration and the degradation removal. In these tasks, polarization serves as a potent cue to disentangle confounding factors such as haze, reflections, and low-light noise. Furthermore, it helps address hardware-induced limitations, including spatial aliasing in demosaicing and extreme illumination in high dynamic range (HDR) reconstruction. At the Physical Layer, we examine the recovery of intrinsic scene properties—including Shape from Polarization (SfP) and inverse rendering—and show how deep learning circumvents the ambiguities and ill-posedness inherent in traditional analytical models. At the Semantic Layer, we explore how polarization-enhanced representations bolster performance in high-level vision tasks, such as object detection and segmentation, especially in scenarios where intensity-based features are compromised. Through this systematic analysis, we identify persistent challenges, notably the scarcity of standardized datasets and the structural limitations of standard network architectures in handling highly sensitive physical signals. Finally, we conclude by recommending several key directions, including physics-aware networks and multi-physics simulation, to foster cohesion and accelerate innovation in the field.</p>