Large language models(LLMs) have exhibited notable general-purpose task-solving abilities in language understanding and generation, including processing recommendation tasks. The majority of existing research relies on training-free recommendation models that treat LLMs as reasoning engines and directly given the recommended task response. This approach heavily relies on pre-trained knowledge and may lead to excessive costs. As such, we propose a two-stage fine-tuning framework leveraging LLaMA2 and GPT-4 Knowledge Enhancement for recommendation. In particular, we use GPT-4 Instruction-Following data to tune the LLM in first-stage instruction tuning process, achieving lower training costs and better inference performance. In the second stage, through a elaborately designed prompt template, we fine-tune LLM from the first stage in a few-shot setting by interactive sequences based on user ratings. To validate the effectiveness of our framework, we compare against state-of-the-art baseline methods on benchmark datasets. The results demonstrate that our framework has promising recommendation capabilities. Our experiments are executed on a single RTX4090 with LLaMA2-7B.