Large Language Models (LLMs) exhibit remarkable performance across a range of tasks, including question answering. However, their tendency to hallucinate, sometimes producing misleading or incorrect responses, remains a critical challenge. In this work, we investigate how LLM-generated answers align with structured relational knowledge from Knowledge Graphs (KGs). We analyze the relationship between KG-derived features and question-answering over various open-source LLMs, uncovering correlations between such features and LLM accuracy. Our study focuses on Wikidata, but is generalizable to other KGs. Leveraging these insights, we propose a feature-based probing strategy that identifies challenging questions likely to expose LLM limitations. We utilized these correlated features to perform an evaluation probing KG-LLM alignment to detect weak spots where KG-LLM alignment is poor. Experimental results show that KG features can effectively guide probing. This work highlights the value of using structured knowledge graphs in building more reliable and trustworthy generative AI systems.