To address the challenges of sparse data and high dynamics in Contextual Multi-armed Bandits (CMAB) models for online recommendation, this study introduces a novel Knowledge Graph-driven Thompson Sampling (KG-TS) algorithm within the CMAB framework. This algorithm innovatively constructs a dynamic Knowledge Graph (KG) that links user characteristics to item attributes, converting sequential decision-making into graph structures to explore data relationships and enhance contextual understanding. Additionally, a time-varying reward mechanism dynamically adjusts the edge weights of the KG, enabling more adaptive and timely personalization in recommendations. Theoretical analysis confirms that KG-TS achieves sublinear cumulative regret growth, demonstrating its efficacy in maximizing long-term benefits. Extensive experiments conducted on two public datasets show that our algorithm outperforms existing bandit algorithms by more than doubling the F1 score and reducing the regret value by over 10%, thus affirming its superior effectiveness in the online recommendation domain.