Unsupervised anomaly detection methods commonly assume that training and testing data share identical distributions, an assumption that rarely holds in real-world scenarios due to distribution shift caused by changes in illumination, imaging devices, or environmental conditions. Such distribution shift often leads to significant degradation in model generalization, limiting the practical deployment of Unsupervised anomaly detection systems. To address these challenges, we propose DIRNet, a novel domain-invariant reconstruction framework designed to enhance robustness under distribution shift by explicitly suppressing style-induced noise and promoting structural consistency. DIRNet consists of three key components: a structure-level feature calibration module that modulates channel-wise responses to filter out style-sensitive information, a self-supervised regularization strategy that enforces structure-level consistency, and a domain-aware decoder guided by a global structural prior to mitigate feature misalignment during reconstruction. Extensive experiments on industrial and natural image benchmarks with distribution shift demonstrate that DIRNet consistently outperforms state-of-the-art methods in both anomaly detection accuracy and generalization, proving its effectiveness for real-world applications subject to distribution shift. Code is available at https://github.com/XHD5656123/DiRNet.