The increasing deployment of intelligent systems in high-stakes domains necessitates a deeper understanding of their decision-making. In this paper, we address the challenges posed by the black-box nature of advanced machine learning models and emphasize the importance of interactive explainable intelligent systems that can enhance user comprehension and agency beyond applying static explainable AI methods. Following a design science research approach, we conducted a literature review and several interviews to derive actionable design requirements. On this basis, we propose design principles for developing a system that prioritizes user interaction, diverse explanation formats, and personalization tailored to varying stakeholder needs. A prototypical implementation demonstrates these principles providing insights into their practical application. Our work contributes to the field by offering normative design knowledge and an instantiation of an interactive explainable intelligent system that caters to different user groups while fostering informed decision-making and mitigating biases.