Large Language Models (LLMs) have emerged as powerful tools for interpreting both unstructured and structured inputs, showing potential for enabling natural language interaction across the Business Process Management (BPM) lifecycle. Despite initial experiments with LLMs in BPM tasks, the practical integration of these models into autonomous, context-aware Artificial Intelligence (AI) systems remains largely conceptual and require detailed capability assessments. Visions of AI-augmented Business Process Management Systems (BPMS) propose agents capable of interpreting process semantics, assisting stakeholders through dialogue and grounding their outputs in formalized process knowledge. Yet, a conceptual disconnect persists between the symbolic representations used in semantic BPM and the sub-symbolic mechanisms of LLMs. This paper presents an empirical study that probes this divergence by evaluating the effectiveness of querying RDF-encoded BPMN models with two retrieval-augmented conversational agents powered by OpenAI’s gpt-4.1, interoperating by different means with knowledge graphs maintained on Ontotext’s GraphDB. Natural language prompts are hereby designed in line with the TELeR taxonomy, addressing a diverse set of BPMN-related queries. The GPT-generated responses are assessed against a subset of metrics from the Retrieval Augmented Generation Assessment (RAGAs) framework. While the work advocates the use of semantic graphs as a mediator in LLM-powered BPMS environments, it also shows limitations in relying strictly on LLM's process analysis.