Accurate multi-category brain tumor image segmentation is crucial for early diagnosis. In recent years, deep learning-based methods have boosted the performance of brain tumor segmentation. However, existing standard models struggle to capture sufficient global context information, which may lead to the neglect of important background knowledge of brain tumors or local features during the segmentation process. Moreover, due to the presence of various sub-regions with significant morphological heterogeneity, size, and positional variations across multiple tumor categories, traditional feature extraction methods fail to adequately capture the complex spatial semantic relations in multi-category tumors. This results in insufficient accuracy and robustness of segmentation results. In this paper, we propose a Multi-scale and Cross-category Relation Modeling method to capture enough global context and handle morphological heterogeneity relations across tumor categories. The network consists of two primary modules: the Adaptive Context Attention Multi-scale Feature Learning Module (ACML) and the Multi-category Feature Enhancement Module (MFEM). ACML dynamically focuses on the relation between local features and global dependencies by incorporating attention mechanisms, effectively extracting multi-scale information. MFEM models the structural relations among multi-categories of brain tumors at various levels, highlighting the most crucial features within each category and integrating them into the overall network structure. Comprehensive experiments conducted on the BraTS2019 and BraTS2020 benchmark datasets demonstrate that our method achieves significant improvements over existing state-of-the-art methods.