Learning to coordinate multiple autonomous agents remains a central challenge in artificial intelligence, particularly in dynamic and partially observable environments. Multi-Agent Reinforcement Learning (MARL) offers a promising paradigm for such distributed decision-making, yet its training process often suffers from non-stationarity, unstable convergence, and reward misalignment between individual and collective rewards. These issues are amplified in safety-critical domains, such as the coordination of Connected and Autonomous Vehicles (CAVs) at unsignalized intersections, where agents must negotiate right-of-way decisions under uncertainty. This paper proposes a hybrid framework that combines MARL, implemented via the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, with a metaheuristic optimization supervisor acting as an adaptive reward modulator. The metaheuristic component dynamically tunes the reward-function coefficients during training, effectively reshaping the causal relationships between agent actions and global outcomes. This adaptive calibration enhances learning stability, promotes cooperative behaviors, and accelerates convergence across agents that follow a Centralized Training and Decentralized Execution (CTDE) scheme, thus enabling adaptation and stable coordination in multi-agent traffic systems.