Numerous cyber-physical systems must self-adapt in order to satisfy complex functional and non-functional requirements despite the uncertainty and change present in their operational environments. Recent advances in Reinforcement Learning (RL) have enabled efficient decision-making for self-adaptation. However, these approaches often suffer from learning bias, potentially leading to sub-optimal or even infeasible adaptations. Conversely, quantitative verification (QV) techniques offer strong guarantees for individual requirements but are computationally costly. Hybrid RL-QV approaches have demonstrated promise, but currently focus exclusively on quality requirements. Our paper proposes a novel framework that integrates QV with RL, enabling decision-making that optimizes the satisfaction of diverse requirements, including temporal event ordering and quality constraints. We validate our approach in an assistive-care robotics scenario, where a robot must achieve multiple goals in a specific sequence while avoiding obstacles and the supported user. The experimental results demonstrate that our approach achieves a balanced trade-off between safety and efficiency, converging faster to near-optimal solutions than a Q-learning baseline.