Breast tumor segmentation in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) achieves precise delineation of tumor boundaries and subregions by capturing rich tissue heterogeneity information. However, its reliance on contrast agents may cause adverse effects, and the acquisition of complete time-series data involves a complex process. In contrast, current non-contrast image segmentation methods suffer from insufficient accuracy due to the lack of explicit tissue heterogeneity information. To address these limitations, we propose an approach for tumor heterogeneity estimation and segmentation in non-contrast images. The core idea is to extract tissue heterogeneity information from DCE-MRI and transfer it to a non-contrast image segmentation network, achieving tumor segmentation accuracy comparable to DCE-MRI-based methods. Our approach uses a vector quantized-variational autoencoder (VQ-VAE)-based clustering model to transform images into heterogeneity maps, capturing structural features of tumor subregions. These maps serve as the ground truth for training. Then, a heterogeneity information prediction model (HIPM) estimates heterogeneity maps from non-contrast images. These features are utilized as prior information to guide the segmentation network, further improving segmentation accuracy. Experimental results demonstrate that the cluster compactness (CPN) and Davies-Bouldin index (DBN) of the clustering reach approximately 0.05 and 0.001, respectively, indicating high clustering accuracy. Our method provides intuitive visualization of tumor heterogeneity without the need for contrast agents and significantly improves segmentation accuracy, with Dice Similarity Coefficient (DSC), Positive Predictive Value (PPV), and Sensitivity (SEN) increased by 20% compared to other non-contrast image segmentation networks.