This paper presents X-SARF, a semantically-aware neural radiance fields framework for novel view synthesis from sparse X-ray images. The framework introduces two key innovations:(1) Contextual Clustering Sampling (CCS) that performs adaptive local clustering in X-ray foreground regions for focused feature extraction, and (2) X-ray Semantic Data Augmentation (XSDA) that generates meaningful variations in deep feature space tailored to X-ray imaging characteristics. X-SARF employs homoscedastic uncertainty weighting for automatic multi-task loss balancing and explicitly models X-ray penetrative physics by eliminating view-dependent components, focusing solely on density-based attenuation. This design fundamentally differs from existing NeRF variants that directly transfer RGB imaging principles. Experiments on knee and chest datasets demonstrate substantial improvements over state-of-the-art methods, achieving average gains of 10.05 dB in PSNR and 0.20 in SSIM. The proposed method enables high-quality novel view synthesis from as few as a single X-ray image.