This research paper presents a new method for tackling resource allocation challenges in multi-Fog and Cloud systems with varying demands, leveraging reinforcement learning. Our approach is structured in two phases: first, we identify the optimal Fog node for resource allocation, and in the second phase, reinforcement learning is applied to determine the best long-term strategy for whether the selected Fog node should handle tasks locally or offload them to the Cloud. To improve convergence speed, we integrate a Genetic Algorithm (GA) into the reinforcement learning process. The goal is to maximize Fog resource utilization while considering the number of resource blocks and each request’s maximum delay tolerance. Experimental evaluations confirm the effectiveness of our method in enhancing resource allocation within Fog computing environments. We benchmark our approach against pure Reinforcement Learning (RL) approach in dynamic resource allocation scenarios, with results showing that the combined approach outperforms others in speed, resource utilization, and load balancing.