The rapid advancements of large language models (LLMs) have opened new avenues for data processing and knowledge extraction, particularly in the medical domain. This paper investigates the application of LLMs in Traditional Chinese Medicine (TCM), with a focus on enhancing the models’ capabilities in syndrome differentiation thinking tasks. We propose a method that delineates the syndrome differentiation process in TCM into four critical steps: clinical information extraction, pathogenesis inference, syndrome inference, and explanatory summarization, with tailored prompting strategies designed for each step. By integrating Retrieval-Augmented Generation (RAG) with instruction tuning, we generated 800 instruction data entries rich in localized knowledge and instruction tuning of a pre-trained model. Experimental results indicate that our approach significantly improves the models’ performance in TCM syndrome differentiation thinking, achieving top rankings in both the A and B leaderboards, with scores of 45.12 and 44.37, respectively.