We propose SSG-HOI, a unified framework for Human-Object Interaction (HOI) detection that enhances spatial reasoning and semantic alignment through structurally guided visual-text modeling. Existing HOI detectors often suffer from limited spatial awareness and insufficient semantic expressiveness, particularly when relying on fixed language templates or fully learnable queries. To address these issues, our method integrates three key components: (1) a Spatial Prior Injection module that introduces multi-scale anchor-based spatial cues into the query embeddings, providing explicit positional guidance; (2) a Text-Augmented HOI Embedding strategy that represents each interaction category using multiple diverse natural language descriptions, encoded via a frozen CLIP text encoder; and (3) a Text-Guided Semantic Fusion module that aligns vision-language representations via cross-attention, enabling fine-grained interaction reasoning. The model adopts a dual decoder architecture that separates human-object localization from interaction classification, improving modularity and interpretability. Extensive experiments on the HICO-DET (38.55%) and V-COCO (61.3%) benchmarks demonstrate that our method achieves state-of-the-art performance, outperforming prior approaches while maintaining model efficiency. The proposed framework offers a robust, efficient, and generalizable solution for HOI detection.