Current deep learning approaches for medical image synthesis require training multiple specialized models for different modality conversions, leading to inefficient parameter utilization. In this work, we propose a unified text-conditioned latent diffusion framework that achieves one-to-many medical image synthesis through two key innovations: (1) With text-guided dynamic gating, a shared latent space construction using pre-trained modality-specific encoders is proposed, reducing model parameters compared to training several separate models. (2) An adaptive hybrid frequency processor combining wavelet decomposition and Fourier analysis is designed to preserve both local textures and global anatomical structures. Our comprehensive experimental evaluation in various datasets validates that this framework is capable of transforming a single medical imaging modality into multiple target modalities using only one model, surpassing existing methods based on Generative Adversarial Networks and diffusion models in terms of generation quality. The success of this work establishes a new paradigm for efficient multi-modal medical image synthesis through latent space unification and frequency-aware diffusion, significantly advancing the practicality of virtual medical image generation systems.