Graph Transformers (GTs), a specialized variant of Graph Neural Networks (GNNs), have gained attention for their effectiveness in node and graph representation. However, GTs primarily capture simple dyadic relations, missing higher-order relations in graphs. Hypergraphs can naturally represent these complex, high-order relationships, leading to the development of hypergraph transformers. Existing hypergraph transformers, however, focus on semantic feature-based attention, often losing structural attributes. To address this, we propose a Topology-guided Hypergraph Transformer Network (THTN). Our model formulates a graph into a hypergraph while preserving structural integrity, facilitating the learning of higher-order relations within the network. Then THTN introduces a novel structure-aware attention mechanism, identifying the importance of nodes and hyperedges from both semantic and structural perspectives. Additionally, a structural and spatial encoding module incorporates topological and spatial information into node representations. By integrating these modules, THTN captures a range of local and global topological features. Extensive experiments on node classification tasks demonstrate that THTN consistently outperforms existing models, showcasing its effectiveness and potential.