Recently, Fully Convolutional Network (FCN)-based Salient Object Detection (SOD) methods achieve strong benchmark performance. However, robustness in real-world scenarios remains an overlooked challenge. Factors like low illumination, overexposure, noise, and low resolution can introduce domain shifts between benchmark datasets and unseen real-world scenarios, limiting model generalization. To address this, we approach SOD from a domain generalization perspective and propose a Side Layer-based Whitening and Residual Refinement Module (WRRM). WRRM enhances feature robustness by eliminating domain-invariant statistical biases through first-order (centering and scaling) and second-order (decorrelation) whitening, while a residual module implicitly retains detail integrity. This study is the first to explore the interplay between whitening, saliency, and object integrity. Extensive experiments demonstrate that our model consistently outperforms SOTA methods under identical backbone settings across five benchmark datasets, while exhibiting superior robustness in challenging scenarios. Furthermore, the proposed WRRM module is highly portable and can be seamlessly integrated as a plug-in to boost the performance of existing SOD frameworks.