Designing high-quality cyber exercises requires more than technical fidelity, it demands credible narratives, immersive scenarios and expert-driven coordination. In this paper, we present a case study from the 2024 Austrian National Cybersecurity Exercise, exploring how ChatGPT was integrated into scenario design, infrastructure development and content creation. We identify four core quality criteria for cyber exercises (credibility, immersion, technical fidelity and expertise) and examine how AI-supported workflows influenced each criterion. Our findings show that ChatGPT can accelerate ideation, reduce drafting time and increase creative flexibility, particularly during early-stage design. At the same time, we highlight structural limitations, including hallucinations, lack of temporal state awareness and restricted utility in offensive simulation. We argue that AI does not replace human expertise but reshapes its application, moving expert input downstream into validation and strategic alignment. The paper concludes with a set of practical observations derived from hands-on use, offering insights for practitioners seeking to integrate AI tools into complex simulation environments, such as gains in early-phase efficiency, improved change resilience but also above mentioned risks and the continued need for expert validation.