By considering a linear supply chain that involves four stages: Retailer, Wholesaler, Distributor, and Factory, this paper is seeking the reinforcement learning (RL) approach to supply order management. Regarding the fulfilment of orders, this paper uses the Proximal Policy Optimization (PPO) algorithm with the objective of minimizing backorder periods against fill rate and total inventory cost of the supply chain. This study tackles limitations found in prior methods by introducing a fill rate component along with inventory cost into the RL reward function. The goal is to improve customer order fulfilment and order management. The study examines two reward functions using two experimental analyses and finds promising findings. Nevertheless, the total inventory cost of Experiment 2 is higher than the one of Experiment 1; however, it drastically reduces the frequency of backorder periods and improve the fill rate. That shows the need to incorporate fill rate into the rewards function, so that the supply chain performance can be enhanced. Studies show how essential it is for SC managers to find a way between demand fulfilment and carrying cost.