Large Language Models (LLMs) are increasingly used to query knowledge graphs (KGs) due to their strong semantic understanding and extrapolation capabilities compared to traditional approaches. However, when KGs contain sensitive information and users lack local access to generative models, privacy becomes a critical concern. To address this issue, we propose a privacy-aware query generation approach for KGs. Our method identifies sensitive information in the graph based on its structure and omits such values before requesting the LLM to translate natural language questions into Cypher queries. Experimental results show that our approach effectively prevents sensitive data from being transmitted to third-party services, while maintaining a high level of query accuracy.