Predictive business process monitoring focuses on predicting future process trajectories, including next-activity predictions. This is crucial in dynamic environments where processes change or face uncertainty. However, current frameworks often assume a static environment, overlooking dynamic characteristics and concept drifts. This results in catastrophic forgetting, where training while focusing merely on new data distribution negatively impacts the performance on previously learned data distributions. Continual learning addresses, among others, the challenges related to mitigating catastrophic forgetting. This paper proposes a novel approach called Continual Next Activity Prediction with Prompts (CNAPwP) which adapts the DualPrompt algorithm for next-activity prediction to improve accuracy and adaptability while mitigating catastrophic forgetting. New datasets with recurring concept drifts are introduced, alongside a task-specific forgetting metric that measures the prediction accuracy gap between initial and subsequent task encounters. Extensive testing on both synthetic and real-world datasets shows that this approach outperforms five competing methods, demonstrating its potential applicability in real-world scenarios. An open source implementation of our method together with datasets and results are available under: https://github.com/TamaraVerbeek/CNAPwP .