To improve the throughput of Ad Hoc such as Wireless Sensor Networks (WSN) and Internet of Vehicles (IoV), we proposed a Dynamic Frame Slotted ALOHA (DFSA). DFSA combines FSA (Frame Slotted ALOHA) with Q-learning to achieve an optimal way to select time slot and dynamically change the frame length. The Q-learning framework is implemented to dynamically optimize slot Q-values through feedback mechanisms and memory retention systems. Agents maintain two critical memory states: 1) the cumulative count of sequential collision or successful transmission events within the current slot, and 2) the temporal distribution pattern of idle slots observed in the preceding frame. To expedite convergence efficiency, a truncated binary exponential increment algorithm is integrated into the Q-value update process. The simulation results show that the average convergence time and collision number of this algorithm are significantly lower than other ALOHA algorithms, the throughput are higher than others.