Path planning plays a crucial role in optimizing robotic warehouse operations, ensuring efficient and timely robot movement while avoiding collision and congestion. This research focuses on congestion-aware path planning for multiple shelf-carrying mobile robots in robotic mobile fulfillment system. Congestion-aware path planning facilitates the navigation of mobile robots in a proficient manner, circumventing areas with high congestion levels while simultaneously minimizing energy consumption and turns executed. To meet the aforementioned objectives, a reinforcement learning-based path planning algorithm with a dynamic action space is proposed. Simulation results for different cases demonstrate the algorithm’s effectiveness in generating feasible paths, offering potential to improved order fulfillment.