The Internet of Drones (IoD) are rapidly gaining attraction due to their adaptability and diverse range of uses. However, the dynamic and distributed nature of IoD networks presents major security challenges, especially in managing access control. Traditional static access control methods are insufficient for adapting to the constantly changing scenarios and threats in drone operations. To address this limitation, dynamic access control approaches have been explored, where access decisions are based on real-time context. In this paper, we propose an ML-Driven Adaptive Risk-Based Access Control framework for the IoD, which continuously updates access permissions based on fluctuating risk levels from various drone operation parameters. The framework has been implemented, estimating the risk value from contextual features, drone-specific features, and Ground Control Station (GCS) parameters. Based on the estimated risk value, the access decision is made. A comparative analysis was performed between various ML models and their performance was evaluated using standard metrics. The results show that our proposed model is more effective in making accurate access decisions under varying environmental and operational conditions.