In view of the growing significance of artificial intelligence (AI) in healthcare, compliance with the European Union’s AI-Act (AI-Act) is crucial for ensuring patient safety, data security and legal conformity. This research-in-progress paper examines the impact of the AI-Act on the design of healthcare processes that are supported conversational agents. It highlights the challenges arising from the new regulatory requirements of the AI-Act and discusses preliminary approaches for integrating these provisions into healthcare processes. The focus is on analyzing the regulatory effects on the automation of healthcare processes. Our first findings provide a basis for further research and the eventual implementation of the AI-Act’s requirements. The paper underlines the need for clear risk classifications and precise definitions of AI systems, as well as their potential effects on efficiency and patient safety in healthcare.