The determination of 3D human pose and shape finds use in multiple computer vision applications. Although recent methods demonstrate high efficiency in daily scenarios, recovering satisfactory 3D human motion from videos containing camouflaged scenarios (e.g., low contrast, color distortion, and edge-distorting noise) remains challenging. To address this, we propose IRPose, an image restoration-based pose estimation system that implements multi-scale contrastive learning, outperforming traditional video-based approaches. Due to the scarcity of 3D human datasets for camouflaged settings, we also introduce a new dataset as a benchmark which includes a series of camouflaged human images. Our evaluation shows that IRPose considerably outperforms the leading methods. Quantitative evaluations on our HCS dataset show IRPose reduces errors by 4.4% (PA-MPJPE), 15.0% (MPJPE), 13.1% (MPVPE), and 16.5% (Accel) compared to our baseline.