Autonomous navigation for unmanned ground vehicles (UGVs) in complex environments requires robust perception for reliable traversability estimation and path planning. Traditional geometric methods often fail in unstructured terrains, necessitating robust scene understanding. Current 3D semantic segmentation methods are mostly directed towards structured environments and are not generalizable for off-road domains. To address this, we propose modifications to LiDAR semantic segmentation by incorporating spatial context and adding an auxiliary task of learning point-wise true height to capture robust features. Experiments on the Rellis-3D dataset demonstrate that the proposed approach outperforms state-of-the-art methods in segmentation accuracy and adaptability, offering a scalable solution for UGV perception in diverse outdoor environments.