<p>Continual learning empowers models to learn from a continuous stream of data while preserving previously acquired knowledge, effectively addressing the challenge of catastrophic forgetting and preserving original abilities. In this study, we propose a new approach that integrates adapters within the self-attention mechanisms of Vision Transformers to enhance knowledge retention when sequentially adding datasets from different domains. Unlike previous methods, which continue learning with only one dataset, our approach introduces domain-specific output heads and features gating, allowing the model to maintain high accuracy on previously learned tasks while seamlessly incorporating only the essential information of more than one domain. The proposed method is compared to the prominent parameter-efficient-fine-tuning methods in the current state-of-the-art. The result provide further evidence that our method can effectively alleviate the limitation of previous works. Furthermore, we conduct a comparative analysis using three datasets: CIFAR-100, Flowers102, and DTD, each representing a distinct domain, to investigate the impact of task order on model performance. Our findings underscore the critical role of dataset sequencing in shaping learning outcomes, demonstrating that strategic ordering can significantly improve the model’s ability to adapt to evolving data distributions over time while preserving the integrity of previously learned knowledge.</p>