Since the introduction of OpenAI’s ChatGPT, the underlying technology of generative AI and large language models (LLMs) has gained tremendous interest in academia. Researchers began experimenting with LLMs’ capabilities in various domains, including business process modeling. While these works indicate a promising potential of LLMs for this task, they do not consider the number of tokens of the prompting strategies and output formats. However, the token amount is the number one cost driver for LLM-based process modeling. In addition, an efficient representation of the conversation state has not been addressed so far. This paper addresses these concerns and introduces and evaluates an approach for efficient LLM-based conversational process modeling. We have implemented our approach as a publicly available online tool. In our experiments, we observed average input token reductions of 94% compared to an existing tool while maintaining even better levels of correctness. Furthermore, a user study at a public science fair indicates solid numbers for the tool’s usefulness.