Large Language Models (LLM) often struggle with factual accuracy in knowledge-intensive domains like law due to their fixed parametric memory and constrained generalization from pre-training data. These limitations make it difficult to handle rare queries, leading to increased hallucination risks and unreliable outputs. This paper proposes Synergistic-RAG, a context-driven retrieve-and-refine framework designed to mitigate hallucinations in domain-specific LLM applications. First, We employ zero-shot learning mechanism to identify long-tail quer-ies, enabling the LLM to apply targeted strategies for rare and domain-specific queries. We then construct a benchmark of 1,000 legal queries to evaluate the LLM’s ability to address underrepresented issues and reduce hallucinations. Secondly, we design a caching module to store low-reliability queries in fast-access memory, improving retrieval of relevant context for rare queries. Finally, we propose a context-aware knowledge retrieval approach that integrates parametric memory with external knowledge to alleviate false information and repetition bias. Experimental results on four legal tasks show that Synergistic-RAG outperforms 16 general and domain-specific LLM baselines, achieving an average performance improvement of 25.27%, and exceeding GPT-4 by 3.13%. Its lightweight design also significantly improved retrieval efficiency. These findings validate that its effectiveness in mitigating hallucinations and its potential for generalization to other knowledge-intensive domains. The code is available at https://github.com/Luna-Luckly/Synergistic-RAG .