In highly dynamic Internet of Things (IoT) scenarios, particularly in UAV-enabled ad hoc networks, traditional ad hoc routing protocols like AODV often degrade in performance when deployed in highly dynamic IoT environments, especially UAV-based networks, due to frequent topology changes and limited adaptability. This paper proposes QMAODV, a Q-learning-based multipath extension of AODV that enables each node to autonomously learn the best next-hop forwarding strategy based on real-time network feedback such as transmission delay and delivery success. By replacing static, hop-count based probabilistic forwarding with a lightweight reinforcement learning model, QMAODV dynamically adapts to topology variations while preserving the low-overhead structure of AODV. Simulations conducted in UAV-enabled FANET scenarios show that QMAODV outperforms both standard AODV and our prior probabilistic multipath AODV (PMAODV) in terms of packet delivery ratio, end-to-end delay, and throughput, with only marginal overhead. The proposed protocol offers a scalable and effective routing solution for mobility-intensive, infrastructure-less IoT networks.