As the automotive industry advances toward higher levels of autonomy, decision-making frameworks must evolve to address increasingly complex and dynamic environments. This paper presents a novel approach called Multi-Grid Markov Decision Processes (mg-MDP), designed to enhance scalability, robustness, and efficiency in autonomous vehicle decision-making. Building on the foundations of traditional Markov Decision Processes (MDPs), mg-MDP utilize a hierarchical multi-layer grid structure to better represent distinct aspects of the environment. Through extensive simulations, we show that mg-MDP incrementally adjusts decision-making across multiple grid- based layers, efficiently handling dynamic traffic scenarios such as intersections, lane merging, and obstacle avoidance. This approach intends to reduce the computational effort while improving decision accuracy. This paper also discusses how mg-MDP can be applied in Cyber-Physical Systems for better real-world modeling that will leverage intelligent transportation.