Retrieval-Augmented Generation (RAG) has been shown to enhance the performance of large language models (LLMs) for open-domain question answering (QA). However, it still faces challenges in multi-hop reasoning scenarios, which require decomposing complex questions and performing step-by-step retrieval and reasoning. Existing Chain-of-Thought (CoT) decomposition methods often suffer from generating irrelevant content and typically rely heavily on domain-specific demonstrations. In this paper, we propose Rewritable Decomposition-Guided Iterative Retrieval-Augmented Generation (ReD-RAG), a novel framework that first decomposes complex questions into multiple contextual sub-questions with explicit rewriting markers. The framework then iteratively conducts rewriting, retrieval, filtering, and generation to solve each sub-question, ultimately integrating multi-hop information to answer the original question. To support high-quality decomposition, we introduce Multi-Decom, a specialized dataset for fine-tuning the question decomposer. Experimental results on three open-source multi-hop QA datasets show that ReD-RAG significantly improves the accuracy of question answering of LLMs, achieving strong performance on models like ChatGPT, LLaMA3. These findings validate the effectiveness of our approach in improving multi-hop reasoning capabilities.