Recent advances in text-driven image editing using visual language models demonstrate considerable potential. However, purely text-based methods face persistent challenges in fashion image editing, where a semantic gap hinders the accurate expression of complex garment details and the generation of visuals faithful to textual descriptions. While FashionTex pioneered the use of texture images as auxiliary inputs with GANs for local editing, inherent limitations of GANs constrain generative quality. To address these issues, we propose TTEdit, a novel diffusion model-based framework for fashion image editing. TTEdit innovatively integrates dual-modality information from text descriptions and texture images to achieve precise, localized garment edits. Our approach consists of two core components: (1) MCNet, a lightweight and accurate localization module built upon mainstream segmentation models to identify editing regions, and (2) the Texture-Adapter, which injects fabric texture features into the diffusion model via a cross-modal attention mechanism. Extensive experiments demonstrate that TTEdit significantly outperforms existing methods, achieving state-of-the-art results with an FID of 11.55, SSIM of 0.871, and CLIP-I of 0.83. These metrics surpass the previous best method, Insert Anything (FID 11.98, SSIM 0.848, CLIP-I 0.81), confirming our framework’s superior generative quality, fidelity, and controllability.