Multi-hop knowledge graph reasoning usually adopts the reinforcement learning framework, which makes the model’s performance dependent on accurate rewards. Existing methods compute path rewards using the prior knowledge in the knowledge graph, which is inaccurate due to insufficient knowledge. Using LLM’s vast internal knowledge to compute reward can alleviate the problem, but the introduction of LLM also poses the problem of lost semantic information and high time overhead. This paper proposes the Prior Knowledge Augmentation- based Knowledge Graph Reasoning (PKA-KGR). In PKA-KGR, the In- Context Learning-based semantic enhancement strategy can mitigate the loss of path semantics in LLM processing. This strategy enhances the capture of path semantics by focusing the LLM on the structural pat- terns implicit in the triples through the relevant context. In addition, PKA-KGR uses the path importance-based dynamic reward strategy to reduce the high time overhead caused by frequent LLM calls. This strategy dynamically identifies important paths and prioritizes the use of LLM internal knowledge to compute their rewards, rather than using LLM to compute rewards for all paths. Such a dynamic strategy reduces the number of LLM calls while avoiding model performance degradation. Our method has been evaluated on four datasets of different scales, and the results reveal that our method outperforms existing methods.