Industrial IoT (IIoT) networks require real-time, dependable communication to support industrial automation, self-driving equipment, and time-critical data sensing applications. Traditional Medium Access Control (MAC) protocols are insufficient because they primarily rely on half-duplex limitations and cannot adapt to unpredictable large-scale traffic. To date, few studies have addressed the Uplink Unused Period (UUP) problem using Full-Duplex MAC (FD-MAC) operation in wireless local area network environments, which can be found unsuitable in IIoT networks. Because in a multi-hop real-time IIoT network, both Unused Uplink and Downlink Periods (UUDP) appear along multi-flow traffic. This study proposes a multi-dimensional, time-sensitive Full-Duplex MAC protocol that dynamically schedules multi-hop and multi-flow communications among IoT nodes using a reinforcement learning (i.e., Q-learning) technique, named MD-RL MAC. To optimize multi-hop slot allocation, our protocol takes specific deadlines, buffer state, queue length, and flow weights into account. The suggested Q-learning model defines the state space, action space, and reward function to optimize scheduling. Although reinforcement learning adds computing cost, the results show that the trade-off is still reasonable for IIoT devices.