Adaptive cyber-physical systems must cope with changing environments while guaranteeing reliable operation. The Multi-Layered Observer/Controller (MLOC) architecture provides a well-established blueprint for such self-organising systems, yet broader adoption is constrained by two key shortcomings: (i) a static, rule-based decision layer that scales poorly with state-space complexity and (ii) a fixed simulator that cannot model unforeseen dynamics. This paper proposes a lightweight redesign of MLOC’s operational and adaptive layers. First, we propose replacing the XCS Classifier System (XCS) in Layer 1 with reinforcement learning agents, which reduces configuration effort and enables continuous control in high-dimensional domains. Second, we suggest a dynamic Layer 2 simulator that is fine-tuned or fully exchanged whenever an anomaly detector signals model drift. The detector differentiates between simulation mismatch and agent uncertainty, triggering simulator updates or focused agent fine-tuning without breaching MLOC’s safety principle of avoiding exploratory actions in the real system. All components are lightweight, making them suitable for platforms with limited resources. Furthermore, depending on the chosen methods, the models may remain interpretable—preserving one of the key advantages of the original XCS. The redesign preserves MLOC’s hierarchical structure and unifies reactive control, long-term learning, and model maintenance, substantially widening its applicability to complex and uncertain environments.