Medical image segmentation is a critical task in medical image analysis, and accurate segmentation results often require high-quality medical image datasets as a foundation. Due to the challenges in acquiring medical datasets, efficient utilization of existing data has become an urgent problem to address. However, current methods do not fully leverage medical data Existing methods directly embed text using attention mechanisms without performing feature alignment between text and images. To address this, we propose a novel and efficient segmentation model, RUMNet, which introduces a reconstruction attention mechanism for image-text fusion in medical semantic segmentation. We designed a feature fusion module to accommodate the different natures of textual and image information, ensuring the full integration of both modalities at different stages. We evaluate RUMNet on the QaTa-COVID19 and MosMedData + datasets, and experimental results demonstrate that RUMNet achieves superior segmentation performance with fewer parameters. RUMNet, with a total of 24.5M parameters, is trained on two medical datasets that contain both textual and image information. On the QaTa-COVID19 dataset, it achieves a Dice score of 83.82% and an mIoU of 75.16%. On the MosMedData + dataset, it achieves a Dice score of 74.19% and an mIoU of 61.26% .