Knob tuning is an important part of database performance optimization. Due to the high dimensionality and complexity of knobs, traditional manual tuning will consume a lot of time for database administrators, and it becomes unrealistic as the number of database instances increases. With the emergence of machine learning, automatic tuning of database knobs has been achieved, but it still requires a lot of iterations, is time-consuming, and has unsatisfactory database performance. This is mainly because the domain knowledge of database knobs is not fully utilized. Therefore, we design an LLM-guided database knob reinforcement learning tuning system (LGTune), which uses LLM to fully utilize database knob knowledge, recommend appropriate knobs in a workload-adaptive manner, and optimize their value range. At the same time, in order to ensure the tuning effect while accelerating the iteration process, we differentiate knobs according to the modification cost, and then use hierarchical reinforcement learning to explore in the optimized search space to obtain the optimal knobs. Experimental results show that LGTune can find knobs with excellent performance while accelerating the iterative convergence speed. Compared with the latest knob tuning tools, these configurations enable the database to have higher throughput and lower latency.