Passive non-line-of-sight (NLOS) imaging technology reconstructs occluded targets by indirectly capturing reflected and scattered light field information from relay surfaces. While this approach offers deployment flexibility and high concealment without active illumination, its strong dependence on ambient light leads to significant signal attenuation, and scattering noise interference results in low signal-to-noise ratio in reconstructed images, severely limiting imaging quality. To address these challenges, this paper proposes the TY-NLOS network model, an improved Transformer-based architecture. It incorporates an image similarity-driven contrastive learning module to enhance discriminative feature extraction under low-light conditions through adversarial training in feature space. A multi-scale attention refinement mechanism is designed, utilizing cascaded dilated convolutions and channel attention fusion strategies to optimize cross-scale feature weighting. Global-local feature synergy is achieved via deep residual and skip connections. Experimental results on both custom and public datasets demonstrate that the proposed method outperforms existing passive NLOS imaging approaches in PSNR and SSIM.