Brain tumors pose a severe threat to human health, with a high mortality rate of 70%. Accurate segmentation of multi-modality magnetic resonance imaging (MRI) is crucial for precise diagnosis and effective treatment planning. However, existing segmentation methods face challenges due to inconsistent lesion masks across modalities and limited availability of precise annotations in certain sequences. To address these challenges, this paper leverages anatomical Consistency across different MR sequences and incorporates a cross Modality guidance module to effectively utilize partially labeled data for brain tumor Segmentation (CoMoSeg). Experiments were conducted on multi-center datasets, including 3,603 partially labeled samples for training and 584 fully-annotated samples from multi-center datasets for testing. The results showed that CoMoSeg improves the Dice coefficient by 2.5% (p < 0.05) over V-Net, 4.9% over a fine-tuned multi-task model and also 4.6% over Swin-UNETR. By enhancing cross modality consistency and effectively training the segmentation model using partially labeled data, CoMoSeg provides a more robust and generalizable segmentation framework. These advancements improve segmentation accuracy and demonstrate great potential for clinical applications in brain tumor diagnosis.